RepoMirrors
/
mpv
mirror of https://github.com/mpv-player/mpv
1
0
Fork 0
Code Issues Releases Wiki Activity
mpv/demux/demux_timeline.c

702 lines
22 KiB
C
Raw Normal View History

Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
/*
* This file is part of mpv.
*
* mpv is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* mpv is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with mpv. If not, see <http://www.gnu.org/licenses/>.
*/
#include <assert.h>
#include <limits.h>
#include "common/common.h"
#include "common/msg.h"
#include "demux.h"
#include "timeline.h"
#include "stheader.h"
ytdl_hook, edl: implement pseudo-DASH support We use the metadata provided by youtube-dl to sort-of implement fragmented DASH streaming. This is all a bit hacky, but hopefully a makeshift solution until libavformat has proper mechanisms. (Although in danger of being one of those temporary hacks that become permanent.)
2017-01-30 18:38:43 +00:00
#include "stream/stream.h"
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
struct segment {
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
int index; // index into virtual_source.segments[] (and timeline.parts[])
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
double start, end;
double d_start;
ytdl_hook, edl: implement pseudo-DASH support We use the metadata provided by youtube-dl to sort-of implement fragmented DASH streaming. This is all a bit hacky, but hopefully a makeshift solution until libavformat has proper mechanisms. (Although in danger of being one of those temporary hacks that become permanent.)
2017-01-30 18:38:43 +00:00
char *url;
bool lazy;
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
struct demuxer *d;
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
// stream_map[sh_stream.index] = virtual_stream, where sh_stream is a stream
// from the source d, and virtual_stream is a streamexported by the
// timeline demuxer (virtual_stream.sh). It's used to map the streams of the
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
// source onto the set of streams of the virtual timeline.
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
// Uses NULL for streams that do not appear in the virtual timeline.
struct virtual_stream **stream_map;
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
int num_stream_map;
};
// Information for each stream on the virtual timeline. (Mirrors streams
// exposed by demux_timeline.)
struct virtual_stream {
struct sh_stream *sh; // stream exported by demux_timeline
bool selected; // ==demux_stream_is_selected(sh)
int eos_packets; // deal with b-frame delay
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
struct virtual_source *src; // group this stream is part of
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
};
demux_timeline: fix a comment
2020-02-15 12:55:54 +00:00
// This represents a single timeline source. (See timeline.pars[]. For each
// timeline_par struct there is a virtual_source.)
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
struct virtual_source {
demux_edl, cue, mkv: clean up timeline stuff slightly Remove the singly linked list hack, replace it with a slightly more proper data structure. This probably gets rid of a few minor bugs along the way, caused by the awkward nonsensical sharing/duplication of some fields. Another change (because I'm touching everything related to timeline anyway) is that I'm removing the special semantics for parts[num_parts]. This is now strictly out of bounds, and instead of using the start time of the next/beyond-last part, there is an end time field now. Unfortunately, this also requires touching the code for cue and mkv ordered chapters. From some superficial testing, they still seem to mostly work. One observable change is that the "no_chapters" header is per-stream now, which is arguably more correct, and getting the old behavior would require adding code to handle it as special-case, so just adjust ytdl_hook.lua to the new behavior.
2019-01-11 12:52:29 +00:00
struct timeline_par *tl;
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
edl: add mechanism for delay loading streams Add something that will access an URL embedded in EDL only when the track it corresponds to is actually selected. This is meant to help with ytdl_hook.lua and to improve loading speeds. In theory, all this stuff is available to any mpv user, but discourage using it, as it's so specialized towards ytdl_hook.lua, that there's danger we'll just break this once ytdl_hook.lua stops using it, or similar. Mostly untested.
2020-02-15 17:09:28 +00:00
bool dash, no_clip, delay_open;
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
struct segment **segments;
int num_segments;
struct segment *current;
demux_timeline: change virtual_stream array to array of pointers Needed for a failed thing, leaving it anyway because it causes no harm and might be less awkward if struct virtual_stream is possibly extended anyway in the future.
2017-10-24 16:39:00 +00:00
struct virtual_stream **streams;
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
int num_streams;
// Total number of packets received past end of segment. Used
// to be clever about determining when to switch segments.
int eos_packets;
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
bool eof_reached;
double dts; // highest read DTS (or PTS if no DTS available)
bool any_selected; // at least one stream is actually selected
demux_timeline: add heuristic to fix shifted seeks with separate audio If you have a EDL stream with separate sources for audio and video stream (like ytdl_hook now creates), you can get the problem that the video stream seeks to a different position than audio due to different key frame granularity. In particular, if you seek backward, the video might undershoot the seek target by a lot. Then video will resume from an earlier position than audio, and the player plays silence. This is annoying. Fix this by explicitly implementing a heuristic to detect separate audio/video streams, determining where a video seek ends up, and then seeking the audio stream to the video destination. This also makes sure to not seek audio with SEEK_FORWARD, so it will always seek before the video position. Non-precise seeks still skip audio to the video target, so this helps with ensuring that audio is present at the final seek target. The implementation is very annoying, because the only way to determine the seek target is to actually read a packet. Thus a 1-packet queue needs to be added. In theory, we could get the seek target from the index of the video file (especially if it's mp4), but libavformat does not have public API that exports this index, so we're stuck with this roundabout generic method. Note that this is only for non-precise seeks. If precise seeks are done, the problem is handled by the frontend by skipping unwanted video frames. But non-precise seeking should still work. (Personally I prefer non-precise seek mode by default because they're still significantly faster.) It also needs to be said that this is the 4th implementation of this seek adjustment thing in mpv. The 1st implementation is in the frontend (look for MPContext.seek_slave). This works only if the external audio stream is known as such on the frontend level. The 2nd implementation is in the demuxer level packet cache (top of execute_cache_seek()). This is similar to code that any demuxer needs to handle non-precise seeks sufficiently nicely. The 3rd is in demux_mkv.c. Since mkv is an interleaved format, this implementation mostly consists on trying to pick index entries for video packets if a video stream is selected. Maybe these "redundant" implementations could be avoided by exposing separate streams through the demuxer API (and making them individually seekable) or something like this, but this is messy and not without problems for multiple reasons. So for now this commit is the best way to fix the observed behavior.
2019-01-11 14:04:43 +00:00
struct demux_packet *next;
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
};
struct priv {
struct timeline *tl;
demux_timeline: fix another cursed memory management issue The timeline stuff has messed up memory management because there are no clear ownership rules between a some demuxer instances (master or demux_timeline) and the timeline object itself. This is another subtle problem that happened: apparently, demux_timeline.open is supposed to take over ownership of timeline, but only on success. If it fails, it's not supposed to free it. It didn't follow this, which lead to a double-free if demux_timeline.open failed. The failure path in demux.c calls both timeline_destroy() and demux_timeline.close on failure.
2020-02-15 13:07:46 +00:00
bool owns_tl;
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
double duration;
// As the demuxer user sees it.
struct virtual_stream **streams;
int num_streams;
struct virtual_source **sources;
int num_sources;
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
};
demux_timeline: report network speed of slave connections demux_timeline doesn't do any transport accesses itself. The slave demuxers do this (these will actually access the stream layer and perform e.g. network accesses). As a consequence, demux_timeline always reported 0 bytes read, and network speed display didn't work. Fix this by awkwardly reporting the amount of read bytes upwards. This is not very nice, and requires explicit calls whenever the slave "might" have read data. Due to the way the reporting is done, it only works if the slaves do not run demuxer threads, which makes things even less nice. (Fortunately they don't anyway, because it would be a waste of resources.) Some identifiers contain the word "hack" as a warning. Some of the stupidity comes from the fact that demux.c itself resets the stats randomly in order to calculate the bytes_per_second value, which is useless for a slave, but of course is still done, because demux.c itself is not aware of whether it's on the slave or top-level layer. Unfortunately, this must do. In theory, the demuxer thread/cache layer should be separated from demuxer implementations. This would get rid of all the awkwardness and nonsense. For example, the only threading involved would be the caching layer, completely separate from demuxers themselves. It'd be the only thing calculates speed rates for the player frontend, too (instead of doing it for each demuxer, even if unused).
2019-01-05 07:52:41 +00:00
static void update_slave_stats(struct demuxer *demuxer, struct demuxer *slave)
{
demux_report_unbuffered_read_bytes(demuxer, demux_get_bytes_read_hack(slave));
}
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
static bool target_stream_used(struct segment *seg, struct virtual_stream *vs)
demux_timeline: move code around Cosmetic preparation for later changes.
2017-01-30 15:16:13 +00:00
{
for (int n = 0; n < seg->num_stream_map; n++) {
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
if (seg->stream_map[n] == vs)
demux_timeline: move code around Cosmetic preparation for later changes.
2017-01-30 15:16:13 +00:00
return true;
}
return false;
}
// Create mapping from segment streams to virtual timeline streams.
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
static void associate_streams(struct demuxer *demuxer,
struct virtual_source *src,
struct segment *seg)
demux_timeline: move code around Cosmetic preparation for later changes.
2017-01-30 15:16:13 +00:00
{
ytdl_hook, edl: implement pseudo-DASH support We use the metadata provided by youtube-dl to sort-of implement fragmented DASH streaming. This is all a bit hacky, but hopefully a makeshift solution until libavformat has proper mechanisms. (Although in danger of being one of those temporary hacks that become permanent.)
2017-01-30 18:38:43 +00:00
if (!seg->d || seg->stream_map)
return;
demux_timeline: move code around Cosmetic preparation for later changes.
2017-01-30 15:16:13 +00:00
int num_streams = demux_get_num_stream(seg->d);
for (int n = 0; n < num_streams; n++) {
struct sh_stream *sh = demux_get_stream(seg->d, n);
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
struct virtual_stream *other = NULL;
for (int i = 0; i < src->num_streams; i++) {
struct virtual_stream *vs = src->streams[i];
// The stream must always have the same media type. Also, a stream
// can't be assigned multiple times.
if (sh->type != vs->sh->type || target_stream_used(seg, vs))
continue;
// By default pick the first matching stream.
if (!other)
other = vs;
demux_timeline: move code around Cosmetic preparation for later changes.
2017-01-30 15:16:13 +00:00
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
// Matching by demuxer ID is supposedly useful and preferable for
// ordered chapters.
demux: do not make up demuxer_id The demuxer_id (exported in as "src-id" property) is supposed to be the native stream ID, as it exists in the file, or -1 if that does not exist (actually any negative value), or if it is unknown. Until now, an ID was made up if it was missing. That seems like strange non-sense, and I can't find the reason why it was done. But it was probably for convenience by the EDL stuff or so. Stop doing this. Fortunately, the src-id property was documented as being unavailable if the ID is not known. Even the code for this was present, it was just inactive until now. Extend input.rst with some explanations. Also fixing 3 other places where negative demuxer_id was ignored or avoided.
2019-12-03 20:04:53 +00:00
if (sh->demuxer_id >= 0 && sh->demuxer_id == vs->sh->demuxer_id)
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
other = vs;
}
demux_timeline: move code around Cosmetic preparation for later changes.
2017-01-30 15:16:13 +00:00
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
MP_TARRAY_APPEND(seg, seg->stream_map, seg->num_stream_map, other);
demux_timeline: move code around Cosmetic preparation for later changes.
2017-01-30 15:16:13 +00:00
}
}
demux_timeline: cosmetics: move a function Gets rid of a forward declaration.
2016-02-16 20:06:02 +00:00
static void reselect_streams(struct demuxer *demuxer)
{
struct priv *p = demuxer->priv;
for (int n = 0; n < p->num_streams; n++) {
demux_timeline: change virtual_stream array to array of pointers Needed for a failed thing, leaving it anyway because it causes no harm and might be less awkward if struct virtual_stream is possibly extended anyway in the future.
2017-10-24 16:39:00 +00:00
struct virtual_stream *vs = p->streams[n];
demux_timeline: cosmetics: move a function Gets rid of a forward declaration.
2016-02-16 20:06:02 +00:00
vs->selected = demux_stream_is_selected(vs->sh);
}
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
for (int x = 0; x < p->num_sources; x++) {
struct virtual_source *src = p->sources[x];
for (int n = 0; n < src->num_segments; n++) {
struct segment *seg = src->segments[n];
ytdl_hook, edl: implement pseudo-DASH support We use the metadata provided by youtube-dl to sort-of implement fragmented DASH streaming. This is all a bit hacky, but hopefully a makeshift solution until libavformat has proper mechanisms. (Although in danger of being one of those temporary hacks that become permanent.)
2017-01-30 18:38:43 +00:00
if (!seg->d)
continue;
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
for (int i = 0; i < seg->num_stream_map; i++) {
bool selected =
seg->stream_map[i] && seg->stream_map[i]->selected;
// This stops demuxer readahead for inactive segments.
if (!src->current || seg->d != src->current->d)
selected = false;
struct sh_stream *sh = demux_get_stream(seg->d, i);
demuxer_select_track(seg->d, sh, MP_NOPTS_VALUE, selected);
demux_timeline: report network speed of slave connections demux_timeline doesn't do any transport accesses itself. The slave demuxers do this (these will actually access the stream layer and perform e.g. network accesses). As a consequence, demux_timeline always reported 0 bytes read, and network speed display didn't work. Fix this by awkwardly reporting the amount of read bytes upwards. This is not very nice, and requires explicit calls whenever the slave "might" have read data. Due to the way the reporting is done, it only works if the slaves do not run demuxer threads, which makes things even less nice. (Fortunately they don't anyway, because it would be a waste of resources.) Some identifiers contain the word "hack" as a warning. Some of the stupidity comes from the fact that demux.c itself resets the stats randomly in order to calculate the bytes_per_second value, which is useless for a slave, but of course is still done, because demux.c itself is not aware of whether it's on the slave or top-level layer. Unfortunately, this must do. In theory, the demuxer thread/cache layer should be separated from demuxer implementations. This would get rid of all the awkwardness and nonsense. For example, the only threading involved would be the caching layer, completely separate from demuxers themselves. It'd be the only thing calculates speed rates for the player frontend, too (instead of doing it for each demuxer, even if unused).
2019-01-05 07:52:41 +00:00
update_slave_stats(demuxer, seg->d);
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
}
}
bool was_selected = src->any_selected;
src->any_selected = false;
for (int n = 0; n < src->num_streams; n++)
src->any_selected |= src->streams[n]->selected;
if (!was_selected && src->any_selected) {
src->eof_reached = false;
src->dts = MP_NOPTS_VALUE;
demux_timeline: add heuristic to fix shifted seeks with separate audio If you have a EDL stream with separate sources for audio and video stream (like ytdl_hook now creates), you can get the problem that the video stream seeks to a different position than audio due to different key frame granularity. In particular, if you seek backward, the video might undershoot the seek target by a lot. Then video will resume from an earlier position than audio, and the player plays silence. This is annoying. Fix this by explicitly implementing a heuristic to detect separate audio/video streams, determining where a video seek ends up, and then seeking the audio stream to the video destination. This also makes sure to not seek audio with SEEK_FORWARD, so it will always seek before the video position. Non-precise seeks still skip audio to the video target, so this helps with ensuring that audio is present at the final seek target. The implementation is very annoying, because the only way to determine the seek target is to actually read a packet. Thus a 1-packet queue needs to be added. In theory, we could get the seek target from the index of the video file (especially if it's mp4), but libavformat does not have public API that exports this index, so we're stuck with this roundabout generic method. Note that this is only for non-precise seeks. If precise seeks are done, the problem is handled by the frontend by skipping unwanted video frames. But non-precise seeking should still work. (Personally I prefer non-precise seek mode by default because they're still significantly faster.) It also needs to be said that this is the 4th implementation of this seek adjustment thing in mpv. The 1st implementation is in the frontend (look for MPContext.seek_slave). This works only if the external audio stream is known as such on the frontend level. The 2nd implementation is in the demuxer level packet cache (top of execute_cache_seek()). This is similar to code that any demuxer needs to handle non-precise seeks sufficiently nicely. The 3rd is in demux_mkv.c. Since mkv is an interleaved format, this implementation mostly consists on trying to pick index entries for video packets if a video stream is selected. Maybe these "redundant" implementations could be avoided by exposing separate streams through the demuxer API (and making them individually seekable) or something like this, but this is messy and not without problems for multiple reasons. So for now this commit is the best way to fix the observed behavior.
2019-01-11 14:04:43 +00:00
TA_FREEP(&src->next);
demux_timeline: cosmetics: move a function Gets rid of a forward declaration.
2016-02-16 20:06:02 +00:00
}
}
}
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
static void close_lazy_segments(struct demuxer *demuxer,
struct virtual_source *src)
ytdl_hook, edl: implement pseudo-DASH support We use the metadata provided by youtube-dl to sort-of implement fragmented DASH streaming. This is all a bit hacky, but hopefully a makeshift solution until libavformat has proper mechanisms. (Although in danger of being one of those temporary hacks that become permanent.)
2017-01-30 18:38:43 +00:00
{
// unload previous segment
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
for (int n = 0; n < src->num_segments; n++) {
struct segment *seg = src->segments[n];
if (seg != src->current && seg->d && seg->lazy) {
demux_timeline: add heuristic to fix shifted seeks with separate audio If you have a EDL stream with separate sources for audio and video stream (like ytdl_hook now creates), you can get the problem that the video stream seeks to a different position than audio due to different key frame granularity. In particular, if you seek backward, the video might undershoot the seek target by a lot. Then video will resume from an earlier position than audio, and the player plays silence. This is annoying. Fix this by explicitly implementing a heuristic to detect separate audio/video streams, determining where a video seek ends up, and then seeking the audio stream to the video destination. This also makes sure to not seek audio with SEEK_FORWARD, so it will always seek before the video position. Non-precise seeks still skip audio to the video target, so this helps with ensuring that audio is present at the final seek target. The implementation is very annoying, because the only way to determine the seek target is to actually read a packet. Thus a 1-packet queue needs to be added. In theory, we could get the seek target from the index of the video file (especially if it's mp4), but libavformat does not have public API that exports this index, so we're stuck with this roundabout generic method. Note that this is only for non-precise seeks. If precise seeks are done, the problem is handled by the frontend by skipping unwanted video frames. But non-precise seeking should still work. (Personally I prefer non-precise seek mode by default because they're still significantly faster.) It also needs to be said that this is the 4th implementation of this seek adjustment thing in mpv. The 1st implementation is in the frontend (look for MPContext.seek_slave). This works only if the external audio stream is known as such on the frontend level. The 2nd implementation is in the demuxer level packet cache (top of execute_cache_seek()). This is similar to code that any demuxer needs to handle non-precise seeks sufficiently nicely. The 3rd is in demux_mkv.c. Since mkv is an interleaved format, this implementation mostly consists on trying to pick index entries for video packets if a video stream is selected. Maybe these "redundant" implementations could be avoided by exposing separate streams through the demuxer API (and making them individually seekable) or something like this, but this is messy and not without problems for multiple reasons. So for now this commit is the best way to fix the observed behavior.
2019-01-11 14:04:43 +00:00
TA_FREEP(&src->next); // might depend on one of the sub-demuxers
demux: get rid of free_demuxer[_and_stream]() Them being separate is just dumb. Replace them with a single demux_free() function, and free its stream by default. Not freeing the stream is only needed in 1 special case (demux_disc.c), use a special flag to not free the stream in this case.
2018-05-19 12:41:06 +00:00
demux_free(seg->d);
ytdl_hook, edl: implement pseudo-DASH support We use the metadata provided by youtube-dl to sort-of implement fragmented DASH streaming. This is all a bit hacky, but hopefully a makeshift solution until libavformat has proper mechanisms. (Although in danger of being one of those temporary hacks that become permanent.)
2017-01-30 18:38:43 +00:00
seg->d = NULL;
}
}
}
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
static void reopen_lazy_segments(struct demuxer *demuxer,
struct virtual_source *src)
ytdl_hook, edl: implement pseudo-DASH support We use the metadata provided by youtube-dl to sort-of implement fragmented DASH streaming. This is all a bit hacky, but hopefully a makeshift solution until libavformat has proper mechanisms. (Although in danger of being one of those temporary hacks that become permanent.)
2017-01-30 18:38:43 +00:00
{
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
if (src->current->d)
ytdl_hook, edl: implement pseudo-DASH support We use the metadata provided by youtube-dl to sort-of implement fragmented DASH streaming. This is all a bit hacky, but hopefully a makeshift solution until libavformat has proper mechanisms. (Although in danger of being one of those temporary hacks that become permanent.)
2017-01-30 18:38:43 +00:00
return;
edl: add mechanism for delay loading streams Add something that will access an URL embedded in EDL only when the track it corresponds to is actually selected. This is meant to help with ytdl_hook.lua and to improve loading speeds. In theory, all this stuff is available to any mpv user, but discourage using it, as it's so specialized towards ytdl_hook.lua, that there's danger we'll just break this once ytdl_hook.lua stops using it, or similar. Mostly untested.
2020-02-15 17:09:28 +00:00
// Note: in delay_open mode, we must _not_ close segments during demuxing,
// because demuxed packets have demux_packet.codec set to objects owned
// by the segments. Closing them would create dangling pointers.
if (!src->delay_open)
close_lazy_segments(demuxer, src);
ytdl_hook, edl: implement pseudo-DASH support We use the metadata provided by youtube-dl to sort-of implement fragmented DASH streaming. This is all a bit hacky, but hopefully a makeshift solution until libavformat has proper mechanisms. (Although in danger of being one of those temporary hacks that become permanent.)
2017-01-30 18:38:43 +00:00
struct demuxer_params params = {
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
.init_fragment = src->tl->init_fragment,
edl: add mechanism for delay loading streams Add something that will access an URL embedded in EDL only when the track it corresponds to is actually selected. This is meant to help with ytdl_hook.lua and to improve loading speeds. In theory, all this stuff is available to any mpv user, but discourage using it, as it's so specialized towards ytdl_hook.lua, that there's danger we'll just break this once ytdl_hook.lua stops using it, or similar. Mostly untested.
2020-02-15 17:09:28 +00:00
.skip_lavf_probing = src->tl->dash,
stream, demux: redo origin policy thing mpv has a very weak and very annoying policy that determines whether a playlist should be used or not. For example, if you play a remote playlist, you usually don't want it to be able to read local filesystem entries. (Although for a media player the impact is small I guess.) It's weak and annoying as in that it does not prevent certain cases which could be interpreted as bad in some cases, such as allowing playlists on the local filesystem to reference remote URLs. It probably barely makes sense, but we just want to exclude some other "definitely not a good idea" things, all while playlists generally just work, so whatever. The policy is: - from the command line anything is played - local playlists can reference anything except "unsafe" streams ("unsafe" means special stream inputs like libavfilter graphs) - remote playlists can reference only remote URLs - things like "memory://" and archives are "transparent" to this This commit does... something. It replaces the weird stream flags with a slightly clearer "origin" value, which is now consequently passed down and used everywhere. It fixes some deviations from the described policy. I wanted to force archives to reference only content within them, but this would probably have been more complicated (or required different abstractions), and I'm too lazy to figure it out, so archives are now "transparent" (playlists within archives behave the same outside). There may be a lot of bugs in this. This is unfortunately a very noisy commit because: - every stream open call now needs to pass the origin - so does every demuxer open call (=> params param. gets mandatory) - most stream were changed to provide the "origin" value - the origin value needed to be passed along in a lot of places - I was too lazy to split the commit Fixes: #7274
2019-12-20 08:41:42 +00:00
.stream_flags = demuxer->stream_origin,
ytdl_hook, edl: implement pseudo-DASH support We use the metadata provided by youtube-dl to sort-of implement fragmented DASH streaming. This is all a bit hacky, but hopefully a makeshift solution until libavformat has proper mechanisms. (Although in danger of being one of those temporary hacks that become permanent.)
2017-01-30 18:38:43 +00:00
};
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
src->current->d = demux_open_url(src->current->url, &params,
demuxer->cancel, demuxer->global);
if (!src->current->d && !demux_cancel_test(demuxer))
ytdl_hook, edl: implement pseudo-DASH support We use the metadata provided by youtube-dl to sort-of implement fragmented DASH streaming. This is all a bit hacky, but hopefully a makeshift solution until libavformat has proper mechanisms. (Although in danger of being one of those temporary hacks that become permanent.)
2017-01-30 18:38:43 +00:00
MP_ERR(demuxer, "failed to load segment\n");
demux: refactor cache range init/deinit Remove the duplicated creation of the first range. Explicitly destroy ranges, including the last one on final deinit. It looks like this also fixes a leak of removed range structs, which was never noticed because they're so small, and were freed on final deinit due to having the demuxer as talloc parent. This improves upon the previous commit too (that change should have been part of it I guess). Sub-demuxers (demux_timeline only) now automatically don't use the cache (like it was intended by the previous commit). The cache is "initialized" (or disabled) last in the recursive call chain, which is messy, but this sub demuxer stuff FUCKING SUCKS, as mentioned in the previous commit message. This would be no problem if the caching layer and actual demuxer implementations were separate. Most of this change has no purpose. Might make (de-)initialization of further cache exerpiments simpler.
2019-06-08 18:32:15 +00:00
if (src->current->d)
demux_timeline: report network speed of slave connections demux_timeline doesn't do any transport accesses itself. The slave demuxers do this (these will actually access the stream layer and perform e.g. network accesses). As a consequence, demux_timeline always reported 0 bytes read, and network speed display didn't work. Fix this by awkwardly reporting the amount of read bytes upwards. This is not very nice, and requires explicit calls whenever the slave "might" have read data. Due to the way the reporting is done, it only works if the slaves do not run demuxer threads, which makes things even less nice. (Fortunately they don't anyway, because it would be a waste of resources.) Some identifiers contain the word "hack" as a warning. Some of the stupidity comes from the fact that demux.c itself resets the stats randomly in order to calculate the bytes_per_second value, which is useless for a slave, but of course is still done, because demux.c itself is not aware of whether it's on the slave or top-level layer. Unfortunately, this must do. In theory, the demuxer thread/cache layer should be separated from demuxer implementations. This would get rid of all the awkwardness and nonsense. For example, the only threading involved would be the caching layer, completely separate from demuxers themselves. It'd be the only thing calculates speed rates for the player frontend, too (instead of doing it for each demuxer, even if unused).
2019-01-05 07:52:41 +00:00
update_slave_stats(demuxer, src->current->d);
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
associate_streams(demuxer, src, src->current);
ytdl_hook, edl: implement pseudo-DASH support We use the metadata provided by youtube-dl to sort-of implement fragmented DASH streaming. This is all a bit hacky, but hopefully a makeshift solution until libavformat has proper mechanisms. (Although in danger of being one of those temporary hacks that become permanent.)
2017-01-30 18:38:43 +00:00
}
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
static void switch_segment(struct demuxer *demuxer, struct virtual_source *src,
struct segment *new, double start_pts, int flags,
bool init)
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
{
demux: drop redundant SEEK_BACKWARD flag Seems like most code dealing with this was for setting it in redundant cases. Now SEEK_BACKWARD is redundant, and SEEK_FORWARD is the odd one out. Also fix that SEEK_FORWARD was not correctly unset in try_seek_cache(). In demux_mkv_seek(), make the arbitrary decision that a video stream is not required for the subtitle prefetch logic to be active. We might want subtitles with long duration even with audio only playback, or if the file is used as external subtitle.
2017-10-23 17:05:39 +00:00
if (!(flags & SEEK_FORWARD))
flags |= SEEK_HR;
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
MP_VERBOSE(demuxer, "switch to segment %d\n", new->index);
demux_timeline: report network speed of slave connections demux_timeline doesn't do any transport accesses itself. The slave demuxers do this (these will actually access the stream layer and perform e.g. network accesses). As a consequence, demux_timeline always reported 0 bytes read, and network speed display didn't work. Fix this by awkwardly reporting the amount of read bytes upwards. This is not very nice, and requires explicit calls whenever the slave "might" have read data. Due to the way the reporting is done, it only works if the slaves do not run demuxer threads, which makes things even less nice. (Fortunately they don't anyway, because it would be a waste of resources.) Some identifiers contain the word "hack" as a warning. Some of the stupidity comes from the fact that demux.c itself resets the stats randomly in order to calculate the bytes_per_second value, which is useless for a slave, but of course is still done, because demux.c itself is not aware of whether it's on the slave or top-level layer. Unfortunately, this must do. In theory, the demuxer thread/cache layer should be separated from demuxer implementations. This would get rid of all the awkwardness and nonsense. For example, the only threading involved would be the caching layer, completely separate from demuxers themselves. It'd be the only thing calculates speed rates for the player frontend, too (instead of doing it for each demuxer, even if unused).
2019-01-05 07:52:41 +00:00
if (src->current && src->current->d)
update_slave_stats(demuxer, src->current->d);
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
src->current = new;
reopen_lazy_segments(demuxer, src);
ytdl_hook, edl: implement pseudo-DASH support We use the metadata provided by youtube-dl to sort-of implement fragmented DASH streaming. This is all a bit hacky, but hopefully a makeshift solution until libavformat has proper mechanisms. (Although in danger of being one of those temporary hacks that become permanent.)
2017-01-30 18:38:43 +00:00
if (!new->d)
return;
demux_timeline: disable cache for inactive segments This is achieved indirectly by deslecting all streams for the non- current segment (and if the segment doesn't share the demuxer with the currently active one). Restores functionality added with commit 46bcdb70.
2016-02-16 20:05:18 +00:00
reselect_streams(demuxer);
demux_edl: add no_clip Used by the next commit. It mostly exposes part of mp4_dash functionality. It actually makes little sense other than for ytdl special-use. See next commit.
2019-01-11 11:07:55 +00:00
if (!src->no_clip)
ytdl_hook, edl: implement pseudo-DASH support We use the metadata provided by youtube-dl to sort-of implement fragmented DASH streaming. This is all a bit hacky, but hopefully a makeshift solution until libavformat has proper mechanisms. (Although in danger of being one of those temporary hacks that become permanent.)
2017-01-30 18:38:43 +00:00
demux_set_ts_offset(new->d, new->start - new->d_start);
demux_edl: add no_clip Used by the next commit. It mostly exposes part of mp4_dash functionality. It actually makes little sense other than for ytdl special-use. See next commit.
2019-01-11 11:07:55 +00:00
if (!src->no_clip || !init)
ytdl_hook, edl: implement pseudo-DASH support We use the metadata provided by youtube-dl to sort-of implement fragmented DASH streaming. This is all a bit hacky, but hopefully a makeshift solution until libavformat has proper mechanisms. (Although in danger of being one of those temporary hacks that become permanent.)
2017-01-30 18:38:43 +00:00
demux_seek(new->d, start_pts, flags);
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
for (int n = 0; n < src->num_streams; n++) {
struct virtual_stream *vs = src->streams[n];
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
vs->eos_packets = 0;
}
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
src->eof_reached = false;
src->eos_packets = 0;
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
}
demux_timeline: add heuristic to fix shifted seeks with separate audio If you have a EDL stream with separate sources for audio and video stream (like ytdl_hook now creates), you can get the problem that the video stream seeks to a different position than audio due to different key frame granularity. In particular, if you seek backward, the video might undershoot the seek target by a lot. Then video will resume from an earlier position than audio, and the player plays silence. This is annoying. Fix this by explicitly implementing a heuristic to detect separate audio/video streams, determining where a video seek ends up, and then seeking the audio stream to the video destination. This also makes sure to not seek audio with SEEK_FORWARD, so it will always seek before the video position. Non-precise seeks still skip audio to the video target, so this helps with ensuring that audio is present at the final seek target. The implementation is very annoying, because the only way to determine the seek target is to actually read a packet. Thus a 1-packet queue needs to be added. In theory, we could get the seek target from the index of the video file (especially if it's mp4), but libavformat does not have public API that exports this index, so we're stuck with this roundabout generic method. Note that this is only for non-precise seeks. If precise seeks are done, the problem is handled by the frontend by skipping unwanted video frames. But non-precise seeking should still work. (Personally I prefer non-precise seek mode by default because they're still significantly faster.) It also needs to be said that this is the 4th implementation of this seek adjustment thing in mpv. The 1st implementation is in the frontend (look for MPContext.seek_slave). This works only if the external audio stream is known as such on the frontend level. The 2nd implementation is in the demuxer level packet cache (top of execute_cache_seek()). This is similar to code that any demuxer needs to handle non-precise seeks sufficiently nicely. The 3rd is in demux_mkv.c. Since mkv is an interleaved format, this implementation mostly consists on trying to pick index entries for video packets if a video stream is selected. Maybe these "redundant" implementations could be avoided by exposing separate streams through the demuxer API (and making them individually seekable) or something like this, but this is messy and not without problems for multiple reasons. So for now this commit is the best way to fix the observed behavior.
2019-01-11 14:04:43 +00:00
static bool do_read_next_packet(struct demuxer *demuxer,
struct virtual_source *src)
{
if (src->next)
return 1;
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
struct segment *seg = src->current;
demux_timeline: add heuristic to fix shifted seeks with separate audio If you have a EDL stream with separate sources for audio and video stream (like ytdl_hook now creates), you can get the problem that the video stream seeks to a different position than audio due to different key frame granularity. In particular, if you seek backward, the video might undershoot the seek target by a lot. Then video will resume from an earlier position than audio, and the player plays silence. This is annoying. Fix this by explicitly implementing a heuristic to detect separate audio/video streams, determining where a video seek ends up, and then seeking the audio stream to the video destination. This also makes sure to not seek audio with SEEK_FORWARD, so it will always seek before the video position. Non-precise seeks still skip audio to the video target, so this helps with ensuring that audio is present at the final seek target. The implementation is very annoying, because the only way to determine the seek target is to actually read a packet. Thus a 1-packet queue needs to be added. In theory, we could get the seek target from the index of the video file (especially if it's mp4), but libavformat does not have public API that exports this index, so we're stuck with this roundabout generic method. Note that this is only for non-precise seeks. If precise seeks are done, the problem is handled by the frontend by skipping unwanted video frames. But non-precise seeking should still work. (Personally I prefer non-precise seek mode by default because they're still significantly faster.) It also needs to be said that this is the 4th implementation of this seek adjustment thing in mpv. The 1st implementation is in the frontend (look for MPContext.seek_slave). This works only if the external audio stream is known as such on the frontend level. The 2nd implementation is in the demuxer level packet cache (top of execute_cache_seek()). This is similar to code that any demuxer needs to handle non-precise seeks sufficiently nicely. The 3rd is in demux_mkv.c. Since mkv is an interleaved format, this implementation mostly consists on trying to pick index entries for video packets if a video stream is selected. Maybe these "redundant" implementations could be avoided by exposing separate streams through the demuxer API (and making them individually seekable) or something like this, but this is messy and not without problems for multiple reasons. So for now this commit is the best way to fix the observed behavior.
2019-01-11 14:04:43 +00:00
if (!seg || !seg->d)
return 0;
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
struct demux_packet *pkt = demux_read_any_packet(seg->d);
demux_edl: add no_clip Used by the next commit. It mostly exposes part of mp4_dash functionality. It actually makes little sense other than for ytdl special-use. See next commit.
2019-01-11 11:07:55 +00:00
if (!pkt || (!src->no_clip && pkt->pts >= seg->end))
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
src->eos_packets += 1;
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
demux_timeline: report network speed of slave connections demux_timeline doesn't do any transport accesses itself. The slave demuxers do this (these will actually access the stream layer and perform e.g. network accesses). As a consequence, demux_timeline always reported 0 bytes read, and network speed display didn't work. Fix this by awkwardly reporting the amount of read bytes upwards. This is not very nice, and requires explicit calls whenever the slave "might" have read data. Due to the way the reporting is done, it only works if the slaves do not run demuxer threads, which makes things even less nice. (Fortunately they don't anyway, because it would be a waste of resources.) Some identifiers contain the word "hack" as a warning. Some of the stupidity comes from the fact that demux.c itself resets the stats randomly in order to calculate the bytes_per_second value, which is useless for a slave, but of course is still done, because demux.c itself is not aware of whether it's on the slave or top-level layer. Unfortunately, this must do. In theory, the demuxer thread/cache layer should be separated from demuxer implementations. This would get rid of all the awkwardness and nonsense. For example, the only threading involved would be the caching layer, completely separate from demuxers themselves. It'd be the only thing calculates speed rates for the player frontend, too (instead of doing it for each demuxer, even if unused).
2019-01-05 07:52:41 +00:00
update_slave_stats(demuxer, seg->d);
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
// Test for EOF. Do this here to properly run into EOF even if other
// streams are disabled etc. If it somehow doesn't manage to reach the end
// after demuxing a high (bit arbitrary) number of packets, assume one of
// the streams went EOF early.
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
bool eos_reached = src->eos_packets > 0;
if (eos_reached && src->eos_packets < 100) {
for (int n = 0; n < src->num_streams; n++) {
struct virtual_stream *vs = src->streams[n];
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
if (vs->selected) {
int max_packets = 0;
if (vs->sh->type == STREAM_AUDIO)
max_packets = 1;
if (vs->sh->type == STREAM_VIDEO)
max_packets = 16;
eos_reached &= vs->eos_packets >= max_packets;
}
}
}
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
src->eof_reached = false;
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
if (eos_reached || !pkt) {
talloc_free(pkt);
struct segment *next = NULL;
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
for (int n = 0; n < src->num_segments - 1; n++) {
if (src->segments[n] == seg) {
next = src->segments[n + 1];
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
break;
}
}
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
if (!next) {
src->eof_reached = true;
demux: return packets directly from demuxer instead of using sh_stream Preparation for other potential changes to separate demuxer cache/thread and actual demuxers. Most things are untested, but it seems to work somewhat.
2018-09-07 13:12:24 +00:00
return false;
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
}
switch_segment(demuxer, src, next, next->start, 0, true);
demux: return packets directly from demuxer instead of using sh_stream Preparation for other potential changes to separate demuxer cache/thread and actual demuxers. Most things are untested, but it seems to work somewhat.
2018-09-07 13:12:24 +00:00
return true; // reader will retry
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
}
demux_timeline: fix off by one error, rearrange weird code This code set pkt->stream to a value which I'm not sure whether it's correct. A recent commit overwrote it with a value that is definitely correct. There appears to be an off by one error. No fucking clue whether this was somehow correct, but applying an apparent fix does not seem to break anything, so whatever.
2018-09-07 19:32:01 +00:00
if (pkt->stream < 0 || pkt->stream >= seg->num_stream_map)
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
goto drop;
edl: add mechanism for delay loading streams Add something that will access an URL embedded in EDL only when the track it corresponds to is actually selected. This is meant to help with ytdl_hook.lua and to improve loading speeds. In theory, all this stuff is available to any mpv user, but discourage using it, as it's so specialized towards ytdl_hook.lua, that there's danger we'll just break this once ytdl_hook.lua stops using it, or similar. Mostly untested.
2020-02-15 17:09:28 +00:00
if (!src->no_clip || src->delay_open) {
demux_timeline: don't use segments for DASH Recent regression. Crashes because it sets the segmented flag, without actually setting the fields required for segmentation.
2017-10-25 22:37:34 +00:00
pkt->segmented = true;
ytdl_hook, edl: implement pseudo-DASH support We use the metadata provided by youtube-dl to sort-of implement fragmented DASH streaming. This is all a bit hacky, but hopefully a makeshift solution until libavformat has proper mechanisms. (Although in danger of being one of those temporary hacks that become permanent.)
2017-01-30 18:38:43 +00:00
if (!pkt->codec)
pkt->codec = demux_get_stream(seg->d, pkt->stream)->codec;
edl: add mechanism for delay loading streams Add something that will access an URL embedded in EDL only when the track it corresponds to is actually selected. This is meant to help with ytdl_hook.lua and to improve loading speeds. In theory, all this stuff is available to any mpv user, but discourage using it, as it's so specialized towards ytdl_hook.lua, that there's danger we'll just break this once ytdl_hook.lua stops using it, or similar. Mostly untested.
2020-02-15 17:09:28 +00:00
}
if (!src->no_clip) {
ytdl_hook, edl: implement pseudo-DASH support We use the metadata provided by youtube-dl to sort-of implement fragmented DASH streaming. This is all a bit hacky, but hopefully a makeshift solution until libavformat has proper mechanisms. (Although in danger of being one of those temporary hacks that become permanent.)
2017-01-30 18:38:43 +00:00
if (pkt->start == MP_NOPTS_VALUE || pkt->start < seg->start)
pkt->start = seg->start;
if (pkt->end == MP_NOPTS_VALUE || pkt->end > seg->end)
pkt->end = seg->end;
}
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
struct virtual_stream *vs = seg->stream_map[pkt->stream];
if (!vs)
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
goto drop;
demux_timeline: enable refresh seeks in some situations Play a trick to make the packet pos field monotonically increasing over segment boundaries if the source demuxers return monotonically increasing pos values. This allows the demuxer to uniquely identify packets with the pos field, and can do refresh seeks using that. Normally, the packet pos field is used as a fallback for determining the playback position if the demuxer returns no proper duration. But demux_timeline.c always will, and the packet pos fields usually make no sense in relation to the returned file size anyway if the timeline source demuxers originate from separate streams.
2016-08-07 11:53:34 +00:00
// for refresh seeks, demux.c prefers monotonically increasing packet pos
// since the packet pos is meaningless anyway for timeline, use it
if (pkt->pos >= 0)
pkt->pos |= (seg->index & 0x7FFFULL) << 48;
demux_edl: add no_clip Used by the next commit. It mostly exposes part of mp4_dash functionality. It actually makes little sense other than for ytdl special-use. See next commit.
2019-01-11 11:07:55 +00:00
if (pkt->pts != MP_NOPTS_VALUE && !src->no_clip && pkt->pts >= seg->end) {
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
// Trust the keyframe flag. Might not always be a good idea, but will
// be sufficient at least with mkv. The problem is that this flag is
// not well-defined in libavformat and is container-dependent.
if (pkt->keyframe || vs->eos_packets == INT_MAX) {
vs->eos_packets = INT_MAX;
goto drop;
} else {
vs->eos_packets += 1;
}
}
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
double dts = pkt->dts != MP_NOPTS_VALUE ? pkt->dts : pkt->pts;
if (src->dts == MP_NOPTS_VALUE || (dts != MP_NOPTS_VALUE && dts > src->dts))
src->dts = dts;
demux: return packets directly from demuxer instead of using sh_stream Preparation for other potential changes to separate demuxer cache/thread and actual demuxers. Most things are untested, but it seems to work somewhat.
2018-09-07 13:12:24 +00:00
pkt->stream = vs->sh->index;
demux_timeline: add heuristic to fix shifted seeks with separate audio If you have a EDL stream with separate sources for audio and video stream (like ytdl_hook now creates), you can get the problem that the video stream seeks to a different position than audio due to different key frame granularity. In particular, if you seek backward, the video might undershoot the seek target by a lot. Then video will resume from an earlier position than audio, and the player plays silence. This is annoying. Fix this by explicitly implementing a heuristic to detect separate audio/video streams, determining where a video seek ends up, and then seeking the audio stream to the video destination. This also makes sure to not seek audio with SEEK_FORWARD, so it will always seek before the video position. Non-precise seeks still skip audio to the video target, so this helps with ensuring that audio is present at the final seek target. The implementation is very annoying, because the only way to determine the seek target is to actually read a packet. Thus a 1-packet queue needs to be added. In theory, we could get the seek target from the index of the video file (especially if it's mp4), but libavformat does not have public API that exports this index, so we're stuck with this roundabout generic method. Note that this is only for non-precise seeks. If precise seeks are done, the problem is handled by the frontend by skipping unwanted video frames. But non-precise seeking should still work. (Personally I prefer non-precise seek mode by default because they're still significantly faster.) It also needs to be said that this is the 4th implementation of this seek adjustment thing in mpv. The 1st implementation is in the frontend (look for MPContext.seek_slave). This works only if the external audio stream is known as such on the frontend level. The 2nd implementation is in the demuxer level packet cache (top of execute_cache_seek()). This is similar to code that any demuxer needs to handle non-precise seeks sufficiently nicely. The 3rd is in demux_mkv.c. Since mkv is an interleaved format, this implementation mostly consists on trying to pick index entries for video packets if a video stream is selected. Maybe these "redundant" implementations could be avoided by exposing separate streams through the demuxer API (and making them individually seekable) or something like this, but this is messy and not without problems for multiple reasons. So for now this commit is the best way to fix the observed behavior.
2019-01-11 14:04:43 +00:00
src->next = pkt;
demux: return packets directly from demuxer instead of using sh_stream Preparation for other potential changes to separate demuxer cache/thread and actual demuxers. Most things are untested, but it seems to work somewhat.
2018-09-07 13:12:24 +00:00
return true;
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
drop:
talloc_free(pkt);
demux: return packets directly from demuxer instead of using sh_stream Preparation for other potential changes to separate demuxer cache/thread and actual demuxers. Most things are untested, but it seems to work somewhat.
2018-09-07 13:12:24 +00:00
return true;
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
}
demux_timeline: reorder some functions Move them around in the source code to get rid of the forward declarations. Other than rearranging the lines and removing the 2 forward declarations, there are no other changes at all.
2020-02-15 12:55:37 +00:00
static bool d_read_packet(struct demuxer *demuxer, struct demux_packet **out_pkt)
{
struct priv *p = demuxer->priv;
struct virtual_source *src = NULL;
for (int x = 0; x < p->num_sources; x++) {
struct virtual_source *cur = p->sources[x];
if (!cur->any_selected || cur->eof_reached)
continue;
if (!cur->current)
switch_segment(demuxer, cur, cur->segments[0], 0, 0, true);
if (!cur->any_selected || !cur->current || !cur->current->d)
continue;
if (!src || cur->dts == MP_NOPTS_VALUE ||
(src->dts != MP_NOPTS_VALUE && cur->dts < src->dts))
src = cur;
}
if (!src)
return false;
if (!do_read_next_packet(demuxer, src))
return false;
*out_pkt = src->next;
src->next = NULL;
return true;
}
static void seek_source(struct demuxer *demuxer, struct virtual_source *src,
double pts, int flags)
{
struct segment *new = src->segments[src->num_segments - 1];
for (int n = 0; n < src->num_segments; n++) {
if (pts < src->segments[n]->end) {
new = src->segments[n];
break;
}
}
switch_segment(demuxer, src, new, pts, flags, false);
src->dts = MP_NOPTS_VALUE;
TA_FREEP(&src->next);
}
static void d_seek(struct demuxer *demuxer, double seek_pts, int flags)
{
struct priv *p = demuxer->priv;
seek_pts = seek_pts * ((flags & SEEK_FACTOR) ? p->duration : 1);
flags &= SEEK_FORWARD | SEEK_HR;
// The intention is to seek audio streams to the same target as video
// streams if they are separate streams. Video streams usually have more
// coarse keyframe snapping, which could leave video without audio.
struct virtual_source *master = NULL;
bool has_slaves = false;
for (int x = 0; x < p->num_sources; x++) {
struct virtual_source *src = p->sources[x];
bool any_audio = false, any_video = false;
for (int i = 0; i < src->num_streams; i++) {
struct virtual_stream *str = src->streams[i];
if (str->selected) {
if (str->sh->type == STREAM_VIDEO)
any_video = true;
if (str->sh->type == STREAM_AUDIO)
any_audio = true;
}
}
if (any_video)
master = src;
// A true slave stream is audio-only; this also prevents that the master
// stream is considered a slave stream.
if (any_audio && !any_video)
has_slaves = true;
}
if (!has_slaves)
master = NULL;
if (master) {
seek_source(demuxer, master, seek_pts, flags);
do_read_next_packet(demuxer, master);
if (master->next && master->next->pts != MP_NOPTS_VALUE) {
// Assume we got a seek target. Actually apply the heuristic.
MP_VERBOSE(demuxer, "adjust seek target from %f to %f\n", seek_pts,
master->next->pts);
seek_pts = master->next->pts;
flags &= ~(unsigned)SEEK_FORWARD;
}
}
for (int x = 0; x < p->num_sources; x++) {
struct virtual_source *src = p->sources[x];
if (src != master)
seek_source(demuxer, src, seek_pts, flags);
}
}
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
static void print_timeline(struct demuxer *demuxer)
{
struct priv *p = demuxer->priv;
MP_VERBOSE(demuxer, "Timeline segments:\n");
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
for (int x = 0; x < p->num_sources; x++) {
struct virtual_source *src = p->sources[x];
if (x >= 1)
MP_VERBOSE(demuxer, " --- new parallel stream ---\n");
for (int n = 0; n < src->num_segments; n++) {
struct segment *seg = src->segments[n];
int src_num = n;
for (int i = 0; i < n; i++) {
if (seg->d && src->segments[i]->d == seg->d) {
src_num = i;
break;
}
}
demux_edl, cue, mkv: clean up timeline stuff slightly Remove the singly linked list hack, replace it with a slightly more proper data structure. This probably gets rid of a few minor bugs along the way, caused by the awkward nonsensical sharing/duplication of some fields. Another change (because I'm touching everything related to timeline anyway) is that I'm removing the special semantics for parts[num_parts]. This is now strictly out of bounds, and instead of using the start time of the next/beyond-last part, there is an end time field now. Unfortunately, this also requires touching the code for cue and mkv ordered chapters. From some superficial testing, they still seem to mostly work. One observable change is that the "no_chapters" header is per-stream now, which is arguably more correct, and getting the old behavior would require adding code to handle it as special-case, so just adjust ytdl_hook.lua to the new behavior.
2019-01-11 12:52:29 +00:00
MP_VERBOSE(demuxer, " %2d: %12f - %12f [%12f] (",
n, seg->start, seg->end, seg->d_start);
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
for (int i = 0; i < seg->num_stream_map; i++) {
struct virtual_stream *vs = seg->stream_map[i];
MP_VERBOSE(demuxer, "%s%d", i ? " " : "",
vs ? vs->sh->index : -1);
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
}
demux_edl, cue, mkv: clean up timeline stuff slightly Remove the singly linked list hack, replace it with a slightly more proper data structure. This probably gets rid of a few minor bugs along the way, caused by the awkward nonsensical sharing/duplication of some fields. Another change (because I'm touching everything related to timeline anyway) is that I'm removing the special semantics for parts[num_parts]. This is now strictly out of bounds, and instead of using the start time of the next/beyond-last part, there is an end time field now. Unfortunately, this also requires touching the code for cue and mkv ordered chapters. From some superficial testing, they still seem to mostly work. One observable change is that the "no_chapters" header is per-stream now, which is arguably more correct, and getting the old behavior would require adding code to handle it as special-case, so just adjust ytdl_hook.lua to the new behavior.
2019-01-11 12:52:29 +00:00
MP_VERBOSE(demuxer, ")\n source %d:'%s'\n", src_num, seg->url);
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
}
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
if (src->dash)
MP_VERBOSE(demuxer, " (Using pseudo-DASH mode.)\n");
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
}
MP_VERBOSE(demuxer, "Total duration: %f\n", p->duration);
}
edl: add mechanism for delay loading streams Add something that will access an URL embedded in EDL only when the track it corresponds to is actually selected. This is meant to help with ytdl_hook.lua and to improve loading speeds. In theory, all this stuff is available to any mpv user, but discourage using it, as it's so specialized towards ytdl_hook.lua, that there's danger we'll just break this once ytdl_hook.lua stops using it, or similar. Mostly untested.
2020-02-15 17:09:28 +00:00
// Copy various (not all) metadata fields from src to dst, but try not to
// overwrite fields in dst that are unset in src.
// May keep data from src by reference.
// Imperfect and arbitrary, only suited for EDL stuff.
static void apply_meta(struct sh_stream *dst, struct sh_stream *src)
{
if (src->demuxer_id >= 0)
dst->demuxer_id = src->demuxer_id;
if (src->title)
dst->title = src->title;
if (src->lang)
dst->lang = src->lang;
dst->default_track = src->default_track;
dst->forced_track = src->forced_track;
if (src->hls_bitrate)
dst->hls_bitrate = src->hls_bitrate;
dst->missing_timestamps = src->missing_timestamps;
if (src->attached_picture)
dst->attached_picture = src->attached_picture;
}
demux_edl, cue, mkv: clean up timeline stuff slightly Remove the singly linked list hack, replace it with a slightly more proper data structure. This probably gets rid of a few minor bugs along the way, caused by the awkward nonsensical sharing/duplication of some fields. Another change (because I'm touching everything related to timeline anyway) is that I'm removing the special semantics for parts[num_parts]. This is now strictly out of bounds, and instead of using the start time of the next/beyond-last part, there is an end time field now. Unfortunately, this also requires touching the code for cue and mkv ordered chapters. From some superficial testing, they still seem to mostly work. One observable change is that the "no_chapters" header is per-stream now, which is arguably more correct, and getting the old behavior would require adding code to handle it as special-case, so just adjust ytdl_hook.lua to the new behavior.
2019-01-11 12:52:29 +00:00
static bool add_tl(struct demuxer *demuxer, struct timeline_par *tl)
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
{
struct priv *p = demuxer->priv;
struct virtual_source *src = talloc_ptrtype(p, src);
*src = (struct virtual_source){
.tl = tl,
.dash = tl->dash,
edl: add mechanism for delay loading streams Add something that will access an URL embedded in EDL only when the track it corresponds to is actually selected. This is meant to help with ytdl_hook.lua and to improve loading speeds. In theory, all this stuff is available to any mpv user, but discourage using it, as it's so specialized towards ytdl_hook.lua, that there's danger we'll just break this once ytdl_hook.lua stops using it, or similar. Mostly untested.
2020-02-15 17:09:28 +00:00
.delay_open = tl->delay_open,
demux_edl: add no_clip Used by the next commit. It mostly exposes part of mp4_dash functionality. It actually makes little sense other than for ytdl special-use. See next commit.
2019-01-11 11:07:55 +00:00
.no_clip = tl->no_clip || tl->dash,
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
.dts = MP_NOPTS_VALUE,
};
edl: add mechanism for delay loading streams Add something that will access an URL embedded in EDL only when the track it corresponds to is actually selected. This is meant to help with ytdl_hook.lua and to improve loading speeds. In theory, all this stuff is available to any mpv user, but discourage using it, as it's so specialized towards ytdl_hook.lua, that there's danger we'll just break this once ytdl_hook.lua stops using it, or similar. Mostly untested.
2020-02-15 17:09:28 +00:00
if (!tl->num_parts)
demux_edl, cue, mkv: clean up timeline stuff slightly Remove the singly linked list hack, replace it with a slightly more proper data structure. This probably gets rid of a few minor bugs along the way, caused by the awkward nonsensical sharing/duplication of some fields. Another change (because I'm touching everything related to timeline anyway) is that I'm removing the special semantics for parts[num_parts]. This is now strictly out of bounds, and instead of using the start time of the next/beyond-last part, there is an end time field now. Unfortunately, this also requires touching the code for cue and mkv ordered chapters. From some superficial testing, they still seem to mostly work. One observable change is that the "no_chapters" header is per-stream now, which is arguably more correct, and getting the old behavior would require adding code to handle it as special-case, so just adjust ytdl_hook.lua to the new behavior.
2019-01-11 12:52:29 +00:00
return false;
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
MP_TARRAY_APPEND(p, p->sources, p->num_sources, src);
demux_edl, cue, mkv: clean up timeline stuff slightly Remove the singly linked list hack, replace it with a slightly more proper data structure. This probably gets rid of a few minor bugs along the way, caused by the awkward nonsensical sharing/duplication of some fields. Another change (because I'm touching everything related to timeline anyway) is that I'm removing the special semantics for parts[num_parts]. This is now strictly out of bounds, and instead of using the start time of the next/beyond-last part, there is an end time field now. Unfortunately, this also requires touching the code for cue and mkv ordered chapters. From some superficial testing, they still seem to mostly work. One observable change is that the "no_chapters" header is per-stream now, which is arguably more correct, and getting the old behavior would require adding code to handle it as special-case, so just adjust ytdl_hook.lua to the new behavior.
2019-01-11 12:52:29 +00:00
p->duration = MPMAX(p->duration, tl->parts[tl->num_parts - 1].end);
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
struct demuxer *meta = tl->track_layout;
edl: add mechanism for delay loading streams Add something that will access an URL embedded in EDL only when the track it corresponds to is actually selected. This is meant to help with ytdl_hook.lua and to improve loading speeds. In theory, all this stuff is available to any mpv user, but discourage using it, as it's so specialized towards ytdl_hook.lua, that there's danger we'll just break this once ytdl_hook.lua stops using it, or similar. Mostly untested.
2020-02-15 17:09:28 +00:00
// delay_open streams normally have meta==NULL, and 1 virtual stream
int num_streams = 0;
if (tl->delay_open) {
num_streams = 1;
} else if (meta) {
num_streams = demux_get_num_stream(meta);
}
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
for (int n = 0; n < num_streams; n++) {
edl: add mechanism for delay loading streams Add something that will access an URL embedded in EDL only when the track it corresponds to is actually selected. This is meant to help with ytdl_hook.lua and to improve loading speeds. In theory, all this stuff is available to any mpv user, but discourage using it, as it's so specialized towards ytdl_hook.lua, that there's danger we'll just break this once ytdl_hook.lua stops using it, or similar. Mostly untested.
2020-02-15 17:09:28 +00:00
struct sh_stream *new = NULL;
if (tl->delay_open) {
assert(tl->sh_meta);
new = demux_alloc_sh_stream(tl->sh_meta->type);
new->codec = tl->sh_meta->codec;
demuxer->is_network = true;
demuxer->is_streaming = true;
} else {
struct sh_stream *sh = demux_get_stream(meta, n);
new = demux_alloc_sh_stream(sh->type);
apply_meta(new, sh);
new->codec = sh->codec;
}
if (tl->sh_meta)
apply_meta(new, tl->sh_meta);
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
demux_add_sh_stream(demuxer, new);
demux_timeline: change virtual_stream array to array of pointers Needed for a failed thing, leaving it anyway because it causes no harm and might be less awkward if struct virtual_stream is possibly extended anyway in the future.
2017-10-24 16:39:00 +00:00
struct virtual_stream *vs = talloc_ptrtype(p, vs);
*vs = (struct virtual_stream){
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
.src = src,
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
.sh = new,
};
MP_TARRAY_APPEND(p, p->streams, p->num_streams, vs);
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
assert(demux_get_stream(demuxer, p->num_streams - 1) == new);
MP_TARRAY_APPEND(src, src->streams, src->num_streams, vs);
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
}
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
for (int n = 0; n < tl->num_parts; n++) {
struct timeline_part *part = &tl->parts[n];
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
demux_timeline: disable pointless packet cache for sub-demuxers It seems like there's nothing stopping from sub-demuxers from keeping packets in the cache, even if it's completely pointless. The top-most demuxer (demux_timeline) already takes care of caching, so sub-demuxers only waste space and time with this. Add a function that can disable the packet cache even at runtime and after packets are read. (It's not clear whether it really can happen that packets are read before demux_timeline gets the sub-demuxers, but there's no reason to make it too fragile.) Call it on all sub-demuxers. For this to work, it seems we have to move the code for setting the seekable_cache flag to before demux_timeline is potentially initialized, because otherwise the cache would be reenabled if the demuxer triggering timeline support is a timeline segment itself (e.g. ordered chapters).
2017-12-10 04:07:36 +00:00
// demux_timeline already does caching, doing it for the sub-demuxers
// would be pointless and wasteful.
demux: propagate streaming flag through demux_timeline Before this commit, EDL or CUE files did not properly enable the cache if they were on "slow" media (stream->streaming==true). This happened because the stream is unset for demux_timeline, so the streaming flag could not be queried anymore. Fix this by adding this flag to struct demuxer, and propagate it exactly like the is_network flag. is_network is not used for checking the cache options anymore, and its main function seems to be something else. Normal http streams set the streaming flag already. This should fix #6958.
2019-09-20 15:01:35 +00:00
if (part->source) {
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
demuxer->is_network |= part->source->is_network;
demux: propagate streaming flag through demux_timeline Before this commit, EDL or CUE files did not properly enable the cache if they were on "slow" media (stream->streaming==true). This happened because the stream is unset for demux_timeline, so the streaming flag could not be queried anymore. Fix this by adding this flag to struct demuxer, and propagate it exactly like the is_network flag. is_network is not used for checking the cache options anymore, and its main function seems to be something else. Normal http streams set the streaming flag already. This should fix #6958.
2019-09-20 15:01:35 +00:00
demuxer->is_streaming |= part->source->is_streaming;
}
demux_timeline: disable pointless packet cache for sub-demuxers It seems like there's nothing stopping from sub-demuxers from keeping packets in the cache, even if it's completely pointless. The top-most demuxer (demux_timeline) already takes care of caching, so sub-demuxers only waste space and time with this. Add a function that can disable the packet cache even at runtime and after packets are read. (It's not clear whether it really can happen that packets are read before demux_timeline gets the sub-demuxers, but there's no reason to make it too fragile.) Call it on all sub-demuxers. For this to work, it seems we have to move the code for setting the seekable_cache flag to before demux_timeline is potentially initialized, because otherwise the cache would be reenabled if the demuxer triggering timeline support is a timeline segment itself (e.g. ordered chapters).
2017-12-10 04:07:36 +00:00
edl: add mechanism for delay loading streams Add something that will access an URL embedded in EDL only when the track it corresponds to is actually selected. This is meant to help with ytdl_hook.lua and to improve loading speeds. In theory, all this stuff is available to any mpv user, but discourage using it, as it's so specialized towards ytdl_hook.lua, that there's danger we'll just break this once ytdl_hook.lua stops using it, or similar. Mostly untested.
2020-02-15 17:09:28 +00:00
if (!part->source)
assert(tl->dash || tl->delay_open);
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
struct segment *seg = talloc_ptrtype(src, seg);
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
*seg = (struct segment){
.d = part->source,
ytdl_hook, edl: implement pseudo-DASH support We use the metadata provided by youtube-dl to sort-of implement fragmented DASH streaming. This is all a bit hacky, but hopefully a makeshift solution until libavformat has proper mechanisms. (Although in danger of being one of those temporary hacks that become permanent.)
2017-01-30 18:38:43 +00:00
.url = part->source ? part->source->filename : part->url,
.lazy = !part->source,
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
.d_start = part->source_start,
.start = part->start,
demux_edl, cue, mkv: clean up timeline stuff slightly Remove the singly linked list hack, replace it with a slightly more proper data structure. This probably gets rid of a few minor bugs along the way, caused by the awkward nonsensical sharing/duplication of some fields. Another change (because I'm touching everything related to timeline anyway) is that I'm removing the special semantics for parts[num_parts]. This is now strictly out of bounds, and instead of using the start time of the next/beyond-last part, there is an end time field now. Unfortunately, this also requires touching the code for cue and mkv ordered chapters. From some superficial testing, they still seem to mostly work. One observable change is that the "no_chapters" header is per-stream now, which is arguably more correct, and getting the old behavior would require adding code to handle it as special-case, so just adjust ytdl_hook.lua to the new behavior.
2019-01-11 12:52:29 +00:00
.end = part->end,
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
};
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
associate_streams(demuxer, src, seg);
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
seg->index = n;
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
MP_TARRAY_APPEND(src, src->segments, src->num_segments, seg);
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
}
edl: add mechanism for delay loading streams Add something that will access an URL embedded in EDL only when the track it corresponds to is actually selected. This is meant to help with ytdl_hook.lua and to improve loading speeds. In theory, all this stuff is available to any mpv user, but discourage using it, as it's so specialized towards ytdl_hook.lua, that there's danger we'll just break this once ytdl_hook.lua stops using it, or similar. Mostly untested.
2020-02-15 17:09:28 +00:00
if (tl->track_layout) {
demuxer->is_network |= tl->track_layout->is_network;
demuxer->is_streaming |= tl->track_layout->is_streaming;
}
demux_edl, cue, mkv: clean up timeline stuff slightly Remove the singly linked list hack, replace it with a slightly more proper data structure. This probably gets rid of a few minor bugs along the way, caused by the awkward nonsensical sharing/duplication of some fields. Another change (because I'm touching everything related to timeline anyway) is that I'm removing the special semantics for parts[num_parts]. This is now strictly out of bounds, and instead of using the start time of the next/beyond-last part, there is an end time field now. Unfortunately, this also requires touching the code for cue and mkv ordered chapters. From some superficial testing, they still seem to mostly work. One observable change is that the "no_chapters" header is per-stream now, which is arguably more correct, and getting the old behavior would require adding code to handle it as special-case, so just adjust ytdl_hook.lua to the new behavior.
2019-01-11 12:52:29 +00:00
return true;
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
}
demux_timeline: reorder some functions Move them around in the source code to get rid of the forward declarations. Other than rearranging the lines and removing the 2 forward declarations, there are no other changes at all.
2020-02-15 12:55:37 +00:00
static int d_open(struct demuxer *demuxer, enum demux_check check)
{
struct priv *p = demuxer->priv = talloc_zero(demuxer, struct priv);
p->tl = demuxer->params ? demuxer->params->timeline : NULL;
if (!p->tl || p->tl->num_pars < 1)
return -1;
demuxer->chapters = p->tl->chapters;
demuxer->num_chapters = p->tl->num_chapters;
struct demuxer *meta = p->tl->meta;
edl: add mechanism for delay loading streams Add something that will access an URL embedded in EDL only when the track it corresponds to is actually selected. This is meant to help with ytdl_hook.lua and to improve loading speeds. In theory, all this stuff is available to any mpv user, but discourage using it, as it's so specialized towards ytdl_hook.lua, that there's danger we'll just break this once ytdl_hook.lua stops using it, or similar. Mostly untested.
2020-02-15 17:09:28 +00:00
if (meta) {
demuxer->metadata = meta->metadata;
demuxer->attachments = meta->attachments;
demuxer->num_attachments = meta->num_attachments;
demuxer->editions = meta->editions;
demuxer->num_editions = meta->num_editions;
demuxer->edition = meta->edition;
}
demux_timeline: reorder some functions Move them around in the source code to get rid of the forward declarations. Other than rearranging the lines and removing the 2 forward declarations, there are no other changes at all.
2020-02-15 12:55:37 +00:00
for (int n = 0; n < p->tl->num_pars; n++) {
if (!add_tl(demuxer, p->tl->pars[n]))
return -1;
}
if (!p->num_sources)
return -1;
demuxer->is_network |= p->tl->is_network;
demuxer->is_streaming |= p->tl->is_streaming;
demuxer->duration = p->duration;
print_timeline(demuxer);
demuxer->seekable = true;
demuxer->partially_seekable = false;
edl: add mechanism for delay loading streams Add something that will access an URL embedded in EDL only when the track it corresponds to is actually selected. This is meant to help with ytdl_hook.lua and to improve loading speeds. In theory, all this stuff is available to any mpv user, but discourage using it, as it's so specialized towards ytdl_hook.lua, that there's danger we'll just break this once ytdl_hook.lua stops using it, or similar. Mostly untested.
2020-02-15 17:09:28 +00:00
const char *format_name = "unknown";
if (meta)
format_name = meta->filetype ? meta->filetype : meta->desc->name;
demuxer->filetype = talloc_asprintf(p, "%s/%s", p->tl->format, format_name);
demux_timeline: reorder some functions Move them around in the source code to get rid of the forward declarations. Other than rearranging the lines and removing the 2 forward declarations, there are no other changes at all.
2020-02-15 12:55:37 +00:00
reselect_streams(demuxer);
demux_timeline: fix another cursed memory management issue The timeline stuff has messed up memory management because there are no clear ownership rules between a some demuxer instances (master or demux_timeline) and the timeline object itself. This is another subtle problem that happened: apparently, demux_timeline.open is supposed to take over ownership of timeline, but only on success. If it fails, it's not supposed to free it. It didn't follow this, which lead to a double-free if demux_timeline.open failed. The failure path in demux.c calls both timeline_destroy() and demux_timeline.close on failure.
2020-02-15 13:07:46 +00:00
p->owns_tl = true;
demux_timeline: reorder some functions Move them around in the source code to get rid of the forward declarations. Other than rearranging the lines and removing the 2 forward declarations, there are no other changes at all.
2020-02-15 12:55:37 +00:00
return 0;
}
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
static void d_close(struct demuxer *demuxer)
{
struct priv *p = demuxer->priv;
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
for (int x = 0; x < p->num_sources; x++) {
struct virtual_source *src = p->sources[x];
src->current = NULL;
demux_timeline: add heuristic to fix shifted seeks with separate audio If you have a EDL stream with separate sources for audio and video stream (like ytdl_hook now creates), you can get the problem that the video stream seeks to a different position than audio due to different key frame granularity. In particular, if you seek backward, the video might undershoot the seek target by a lot. Then video will resume from an earlier position than audio, and the player plays silence. This is annoying. Fix this by explicitly implementing a heuristic to detect separate audio/video streams, determining where a video seek ends up, and then seeking the audio stream to the video destination. This also makes sure to not seek audio with SEEK_FORWARD, so it will always seek before the video position. Non-precise seeks still skip audio to the video target, so this helps with ensuring that audio is present at the final seek target. The implementation is very annoying, because the only way to determine the seek target is to actually read a packet. Thus a 1-packet queue needs to be added. In theory, we could get the seek target from the index of the video file (especially if it's mp4), but libavformat does not have public API that exports this index, so we're stuck with this roundabout generic method. Note that this is only for non-precise seeks. If precise seeks are done, the problem is handled by the frontend by skipping unwanted video frames. But non-precise seeking should still work. (Personally I prefer non-precise seek mode by default because they're still significantly faster.) It also needs to be said that this is the 4th implementation of this seek adjustment thing in mpv. The 1st implementation is in the frontend (look for MPContext.seek_slave). This works only if the external audio stream is known as such on the frontend level. The 2nd implementation is in the demuxer level packet cache (top of execute_cache_seek()). This is similar to code that any demuxer needs to handle non-precise seeks sufficiently nicely. The 3rd is in demux_mkv.c. Since mkv is an interleaved format, this implementation mostly consists on trying to pick index entries for video packets if a video stream is selected. Maybe these "redundant" implementations could be avoided by exposing separate streams through the demuxer API (and making them individually seekable) or something like this, but this is messy and not without problems for multiple reasons. So for now this commit is the best way to fix the observed behavior.
2019-01-11 14:04:43 +00:00
TA_FREEP(&src->next);
demux, demux_edl: add extension for tracks sourced from separate streams This commit adds an extension to mpv EDL, which basically allows you to do the same as --audio-file, --external-file, etc. in a single EDL file. This is a relatively quick & dirty implementation. The dirty part lies in the fact that several shortcuts are taken. For example, struct timeline now forms a singly linked list, which is really weird, but also means the other timeline using demuxers (cue, mkv) don't need to be touched. Also, memory management becomes even worse (weird object ownership rules that are just fragile WTFs). There are some other dubious small changes, mostly related to the weird representation of separate streams. demux_timeline.c contains the actual implementation of the separate stream handling. For the most part, most things that used to be on the top level are now in struct virtual_source, of which one for each separate stream exists. This is basically like running multiple demux_edl.c in parallel. Some changes could strictly speaking be split into a separate commit, such as the stream_map type change. Mostly untested. Seems to work for the intended purpose. Potential for regressions for other timeline uses (like ordered chapters) is probably low. One thing which could definitely break and which I didn't test is the pseudo-DASH fragmented EDL code, of which ytdl can trigger various forms in obscure situations. (Uh why don't we have a test suite.) Background: The intention is to use this for the ytdl wrapper. A certain streaming site from a particularly brain damaged and plain evil Silicon Valley company usually provides streams as separate audio and video streams. The ytdl wrapper simply does use audio-add (i.e. adding it as external track, like with --audio-file), which works mostly fine. Unfortunately, mpv manages caching completely separately for external files. This has the following potential problems: 1. Seek ranges are rendered incorrectly. They always use the "main" stream, in this case the video stream. E.g. clicking into a cached range on the OSC could trigger a low level seek if the audio stream is actually not cached at the target position. 2. The stream cache bloats unnecessarily. Each stream may allocate the full configured maximum cache size, which is not what the user intends to do. Cached ranges are not pruned the same way, which creates disjoint cache ranges, which only use memory and won't help with fast seeking or playback. 3. mpv will try to aggressively read from both streams. This is done from different threads, with no regard which stream is more important. So it might happen that one stream starves the other one, especially if they have different bitrates. 4. Every stream will use a separate thread, which is an unnecessary waste of system resources. In theory, the following solutions are available (this commit works towards D): A. Centrally manage reading and caching of all streams. A single thread would do all I/O, and decide from which stream it should read next. As long as the total TCP/socket buffering is not too high, this should be effective to avoid starvation issues. This can also manage the cached ranges better. It would also get rid of the quite useless additional demuxer threads. This solution is conceptually simple, but requires refactoring the entire demuxer middle layer. B. Attempt to coordinate the demuxer threads. This would maintain a shared cache and readahead state to solve the mentioned problems explicitly. While this sounds simple and like an incremental change, it's probably hard to implement, creates more messy special cases, solution A. seems just a better and simpler variant of this. (On the other hand, A. requires refactoring more code.) C. Render an intersection of the seek ranges across all streams. This fixes only problem 1. D. Merge all streams in a dedicated wrapper demuxer. The general demuxer layer remains unchanged, and reading from separate streams is handled as special case. This effectively achieves the same as A. In particular, caching is simply handled by the usual demuxer cache layer, which sees the wrapper demuxer as a single stream of interleaved packets. One implementation variant of this is to reuse the EDL infrastructure, which this commit does. All in all, solution A would be preferable, because it's cleaner and works for all external streams in general. Some previous commit tried to prepare for implementing solution A. This could still happen. But it could take years until this is finally seriously started and finished. In any case, this commit doesn't block or complicate such attempts, which is also why it's the way to go. It's worth mentioning that original mplayer handles external files by creating a wrapper demuxer. This is like a less ideal mixture of A. and D. (The similarity with A. is that extending the mplayer approach to be fully dynamic and without certain disadvantages caused by the wrapper would end up with A. anyway. The similarity with D. is that due to the wrapper, no higher level code needs to be changed.)
2019-01-04 12:09:02 +00:00
close_lazy_segments(demuxer, src);
}
demux_timeline: fix another cursed memory management issue The timeline stuff has messed up memory management because there are no clear ownership rules between a some demuxer instances (master or demux_timeline) and the timeline object itself. This is another subtle problem that happened: apparently, demux_timeline.open is supposed to take over ownership of timeline, but only on success. If it fails, it's not supposed to free it. It didn't follow this, which lead to a double-free if demux_timeline.open failed. The failure path in demux.c calls both timeline_destroy() and demux_timeline.close on failure.
2020-02-15 13:07:46 +00:00
if (p->owns_tl) {
struct demuxer *master = p->tl->demuxer;
timeline_destroy(p->tl);
demux_free(master);
}
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
}
demux: get rid of ->control callback The only thing left is the notification for track switching. Just get rid of that. There's probably no real reason to get rid of control(), but why not. I think I was actually trying to do some real work but fuck that.
2018-09-07 21:10:14 +00:00
static void d_switched_tracks(struct demuxer *demuxer)
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
{
demux: get rid of ->control callback The only thing left is the notification for track switching. Just get rid of that. There's probably no real reason to get rid of control(), but why not. I think I was actually trying to do some real work but fuck that.
2018-09-07 21:10:14 +00:00
reselect_streams(demuxer);
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
}
const demuxer_desc_t demuxer_desc_timeline = {
.name = "timeline",
demux_timeline: slightly improve reported file format Report the underlying demuxer's format. Since there can be many demuxers participating, pick the "main" segment.
2016-02-25 21:49:50 +00:00
.desc = "timeline segments",
demux: return packets directly from demuxer instead of using sh_stream Preparation for other potential changes to separate demuxer cache/thread and actual demuxers. Most things are untested, but it seems to work somewhat.
2018-09-07 13:12:24 +00:00
.read_packet = d_read_packet,
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
.open = d_open,
.close = d_close,
.seek = d_seek,
demux: get rid of ->control callback The only thing left is the notification for track switching. Just get rid of that. There's probably no real reason to get rid of control(), but why not. I think I was actually trying to do some real work but fuck that.
2018-09-07 21:10:14 +00:00
.switched_tracks = d_switched_tracks,
Rewrite ordered chapters and timeline stuff This uses a different method to piece segments together. The old approach basically changes to a new file (with a new start offset) any time a segment ends. This meant waiting for audio/video end on segment end, and then changing to the new segment all at once. It had a very weird impact on the playback core, and some things (like truly gapless segment transitions, or frame backstepping) just didn't work. The new approach adds the demux_timeline pseudo-demuxer, which presents an uniform packet stream from the many segments. This is pretty similar to how ordered chapters are implemented everywhere else. It also reminds of the FFmpeg concat pseudo-demuxer. The "pure" version of this approach doesn't work though. Segments can actually have different codec configurations (different extradata), and subtitles are most likely broken too. (Subtitles have multiple corner cases which break the pure stream-concatenation approach completely.) To counter this, we do two things: - Reinit the decoder with each segment. We go as far as allowing concatenating files with completely different codecs for the sake of EDL (which also uses the timeline infrastructure). A "lighter" approach would try to make use of decoder mechanism to update e.g. the extradata, but that seems fragile. - Clip decoded data to segment boundaries. This is equivalent to normal playback core mechanisms like hr-seek, but now the playback core doesn't need to care about these things. These two mechanisms are equivalent to what happened in the old implementation, except they don't happen in the playback core anymore. In other words, the playback core is completely relieved from timeline implementation details. (Which honestly is exactly what I'm trying to do here. I don't think ordered chapter behavior deserves improvement, even if it's bad - but I want to get it out from the playback core.) There is code duplication between audio and video decoder common code. This is awful and could be shareable - but this will happen later. Note that the audio path has some code to clip audio frames for the purpose of codec preroll/gapless handling, but it's not shared as sharing it would cause more pain than it would help.
2016-02-15 20:04:07 +00:00
};