RepoMirrors
/
mpv
mirror of https://github.com/mpv-player/mpv
1
0
Fork 0
Code Issues Releases Wiki Activity
mpv/demux/demux_libarchive.c

102 lines
2.8 KiB
C
Raw Normal View History

stream: libarchive wrapper for reading compressed archives This works similar to the existing .rar support, but uses libarchive. libarchive supports a number of formats, including zip and (most of) rar. Unfortunately, seeking does not work too well. Most libarchive readers do not support seeking, so it's emulated by skipping data until the target position. On backwards seek, the file is reopened. This works fine on a local machine (and if the file is not too large), but will perform not so well over network connection. This is disabled by default for now. One reason is that we try libarchive on every file we open, before trying libavformat, and I'm not sure if I trust libarchive that much yet. Another reason is that this breaks multivolume rar support. While libarchive supports seeking in rar, and (probably) supports multivolume archive, our support of libarchive (probably) does not. I don't care about multivolume rar, but vocal users do.
2015-08-16 22:55:26 +00:00
/*
* This file is part of mpv.
*
* mpv is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* mpv is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with mpv. If not, see <http://www.gnu.org/licenses/>.
*/
#include <archive.h>
#include <archive_entry.h>
#include "common/common.h"
#include "common/playlist.h"
#include "stream/stream.h"
demux: sort filenames naturally when playing a directory / archive
2018-07-27 03:01:35 +00:00
#include "misc/natural_sort.h"
stream: libarchive wrapper for reading compressed archives This works similar to the existing .rar support, but uses libarchive. libarchive supports a number of formats, including zip and (most of) rar. Unfortunately, seeking does not work too well. Most libarchive readers do not support seeking, so it's emulated by skipping data until the target position. On backwards seek, the file is reopened. This works fine on a local machine (and if the file is not too large), but will perform not so well over network connection. This is disabled by default for now. One reason is that we try libarchive on every file we open, before trying libavformat, and I'm not sure if I trust libarchive that much yet. Another reason is that this breaks multivolume rar support. While libarchive supports seeking in rar, and (probably) supports multivolume archive, our support of libarchive (probably) does not. I don't care about multivolume rar, but vocal users do.
2015-08-16 22:55:26 +00:00
#include "demux.h"
#include "stream/stream_libarchive.h"
static int cmp_filename(const void *a, const void *b)
{
demux: sort filenames naturally when playing a directory / archive
2018-07-27 03:01:35 +00:00
return mp_natural_sort_cmp(*(char **)a, *(char **)b);
stream: libarchive wrapper for reading compressed archives This works similar to the existing .rar support, but uses libarchive. libarchive supports a number of formats, including zip and (most of) rar. Unfortunately, seeking does not work too well. Most libarchive readers do not support seeking, so it's emulated by skipping data until the target position. On backwards seek, the file is reopened. This works fine on a local machine (and if the file is not too large), but will perform not so well over network connection. This is disabled by default for now. One reason is that we try libarchive on every file we open, before trying libavformat, and I'm not sure if I trust libarchive that much yet. Another reason is that this breaks multivolume rar support. While libarchive supports seeking in rar, and (probably) supports multivolume archive, our support of libarchive (probably) does not. I don't care about multivolume rar, but vocal users do.
2015-08-16 22:55:26 +00:00
}
static int open_file(struct demuxer *demuxer, enum demux_check check)
{
demux, stream: add option to prevent opening referenced files Quite irresponsibly hacked together. Sue me.
2016-12-04 22:15:31 +00:00
if (!demuxer->access_references)
return -1;
demux_libarchive: open flat compressed files Things like .gz etc., which have no real file header. A mixed bag, because it e.g. tends to misdetect mp3 files as compressed files or something (of course it has no mp3 support - I don't know as what it detects them). But requested by someone (or maybe not, I'm not sure how to interpret that).
2015-08-17 21:59:44 +00:00
int flags = 0;
demux_libarchive: don't allow probing to read unlimited data Instead, allow reading 2KB only. This seems to be sufficient for libarchive to recognize zip, 7z, rar, tar. Good enough. This is implemented by creating an in-memory stream with a copy of the file header. If libarchive succeeds opening this, the actual stream is opened. Allowing unlimited reading could break unseekable streams, such as playing from http servers with no range request support or pipes. Also, we try not to read too much data in the first probe pass. Some slow network streams like shoutcast services could make probing much slower if we allow it to read too much. In the second probing pass, actually allow 200KB.
2015-08-24 20:21:36 +00:00
int probe_size = STREAM_BUFFER_SIZE;
if (check <= DEMUX_CHECK_REQUEST) {
demux_libarchive: open flat compressed files Things like .gz etc., which have no real file header. A mixed bag, because it e.g. tends to misdetect mp3 files as compressed files or something (of course it has no mp3 support - I don't know as what it detects them). But requested by someone (or maybe not, I'm not sure how to interpret that).
2015-08-17 21:59:44 +00:00
flags |= MP_ARCHIVE_FLAG_UNSAFE;
demux_libarchive: don't allow probing to read unlimited data Instead, allow reading 2KB only. This seems to be sufficient for libarchive to recognize zip, 7z, rar, tar. Good enough. This is implemented by creating an in-memory stream with a copy of the file header. If libarchive succeeds opening this, the actual stream is opened. Allowing unlimited reading could break unseekable streams, such as playing from http servers with no range request support or pipes. Also, we try not to read too much data in the first probe pass. Some slow network streams like shoutcast services could make probing much slower if we allow it to read too much. In the second probing pass, actually allow 200KB.
2015-08-24 20:21:36 +00:00
probe_size *= 100;
}
stream: turn into a ring buffer, make size configurable In some corner cases (see #6802), it can be beneficial to use a larger stream buffer size. Use this as argument to rewrite everything for no reason. Turn stream.c itself into a ring buffer, with configurable size. The latter would have been easily achievable with minimal changes, and the ring buffer is the hard part. There is no reason to have a ring buffer at all, except possibly if ffmpeg don't fix their awful mp4 demuxer, and some subtle issues with demux_mkv.c wanting to seek back by small offsets (the latter was handled with small stream_peek() calls, which are unneeded now). In addition, this turns small forward seeks into reads (where data is simply skipped). Before this commit, only stream_skip() did this (which also mean that stream_skip() simply calls stream_seek() now). Replace all stream_peek() calls with something else (usually stream_read_peek()). The function was a problem, because it returned a pointer to the internal buffer, which is now a ring buffer with wrapping. The new function just copies the data into a buffer, and in some cases requires callers to dynamically allocate memory. (The most common case, demux_lavf.c, required a separate buffer allocation anyway due to FFmpeg "idiosyncrasies".) This is the bulk of the demuxer_* changes. I'm not happy with this. There still isn't a good reason why there should be a ring buffer, that is complex, and most of the time just wastes half of the available memory. Maybe another rewrite soon. It also contains bugs; you're an alpha tester now.
2019-11-06 20:36:02 +00:00
void *probe = ta_alloc_size(NULL, probe_size);
if (!probe)
demux_libarchive: don't allow probing to read unlimited data Instead, allow reading 2KB only. This seems to be sufficient for libarchive to recognize zip, 7z, rar, tar. Good enough. This is implemented by creating an in-memory stream with a copy of the file header. If libarchive succeeds opening this, the actual stream is opened. Allowing unlimited reading could break unseekable streams, such as playing from http servers with no range request support or pipes. Also, we try not to read too much data in the first probe pass. Some slow network streams like shoutcast services could make probing much slower if we allow it to read too much. In the second probing pass, actually allow 200KB.
2015-08-24 20:21:36 +00:00
return -1;
stream: turn into a ring buffer, make size configurable In some corner cases (see #6802), it can be beneficial to use a larger stream buffer size. Use this as argument to rewrite everything for no reason. Turn stream.c itself into a ring buffer, with configurable size. The latter would have been easily achievable with minimal changes, and the ring buffer is the hard part. There is no reason to have a ring buffer at all, except possibly if ffmpeg don't fix their awful mp4 demuxer, and some subtle issues with demux_mkv.c wanting to seek back by small offsets (the latter was handled with small stream_peek() calls, which are unneeded now). In addition, this turns small forward seeks into reads (where data is simply skipped). Before this commit, only stream_skip() did this (which also mean that stream_skip() simply calls stream_seek() now). Replace all stream_peek() calls with something else (usually stream_read_peek()). The function was a problem, because it returned a pointer to the internal buffer, which is now a ring buffer with wrapping. The new function just copies the data into a buffer, and in some cases requires callers to dynamically allocate memory. (The most common case, demux_lavf.c, required a separate buffer allocation anyway due to FFmpeg "idiosyncrasies".) This is the bulk of the demuxer_* changes. I'm not happy with this. There still isn't a good reason why there should be a ring buffer, that is complex, and most of the time just wastes half of the available memory. Maybe another rewrite soon. It also contains bugs; you're an alpha tester now.
2019-11-06 20:36:02 +00:00
int probe_got = stream_read_peek(demuxer->stream, probe, probe_size);
stream: create memory streams in more straightforward way Instead of having to rely on the protocol matching, make a function that creates a stream from a stream_info_t directly. Instead of going through a weird indirection with STREAM_CTRL, add a direct argument for non-text arguments to the open callback. Instead of creating a weird dummy mpv_global, just pass an existing one from all callers. (The latter one is just an artifact from the past, where mpv_global wasn't available everywhere.) Actually I just wanted a function that creates a stream without any of that bullshit. This goal was slightly missed, since you still need this heavy "constructor" just to setup a shitty struct with some shitty callbacks.
2019-06-19 14:48:46 +00:00
struct stream *probe_stream =
stream: turn into a ring buffer, make size configurable In some corner cases (see #6802), it can be beneficial to use a larger stream buffer size. Use this as argument to rewrite everything for no reason. Turn stream.c itself into a ring buffer, with configurable size. The latter would have been easily achievable with minimal changes, and the ring buffer is the hard part. There is no reason to have a ring buffer at all, except possibly if ffmpeg don't fix their awful mp4 demuxer, and some subtle issues with demux_mkv.c wanting to seek back by small offsets (the latter was handled with small stream_peek() calls, which are unneeded now). In addition, this turns small forward seeks into reads (where data is simply skipped). Before this commit, only stream_skip() did this (which also mean that stream_skip() simply calls stream_seek() now). Replace all stream_peek() calls with something else (usually stream_read_peek()). The function was a problem, because it returned a pointer to the internal buffer, which is now a ring buffer with wrapping. The new function just copies the data into a buffer, and in some cases requires callers to dynamically allocate memory. (The most common case, demux_lavf.c, required a separate buffer allocation anyway due to FFmpeg "idiosyncrasies".) This is the bulk of the demuxer_* changes. I'm not happy with this. There still isn't a good reason why there should be a ring buffer, that is complex, and most of the time just wastes half of the available memory. Maybe another rewrite soon. It also contains bugs; you're an alpha tester now.
2019-11-06 20:36:02 +00:00
stream_memory_open(demuxer->global, probe, probe_got);
demux_libarchive: don't allow probing to read unlimited data Instead, allow reading 2KB only. This seems to be sufficient for libarchive to recognize zip, 7z, rar, tar. Good enough. This is implemented by creating an in-memory stream with a copy of the file header. If libarchive succeeds opening this, the actual stream is opened. Allowing unlimited reading could break unseekable streams, such as playing from http servers with no range request support or pipes. Also, we try not to read too much data in the first probe pass. Some slow network streams like shoutcast services could make probing much slower if we allow it to read too much. In the second probing pass, actually allow 200KB.
2015-08-24 20:21:36 +00:00
struct mp_archive *mpa = mp_archive_new(mp_null_log, probe_stream, flags);
bool ok = !!mpa;
free_stream(probe_stream);
mp_archive_free(mpa);
stream: turn into a ring buffer, make size configurable In some corner cases (see #6802), it can be beneficial to use a larger stream buffer size. Use this as argument to rewrite everything for no reason. Turn stream.c itself into a ring buffer, with configurable size. The latter would have been easily achievable with minimal changes, and the ring buffer is the hard part. There is no reason to have a ring buffer at all, except possibly if ffmpeg don't fix their awful mp4 demuxer, and some subtle issues with demux_mkv.c wanting to seek back by small offsets (the latter was handled with small stream_peek() calls, which are unneeded now). In addition, this turns small forward seeks into reads (where data is simply skipped). Before this commit, only stream_skip() did this (which also mean that stream_skip() simply calls stream_seek() now). Replace all stream_peek() calls with something else (usually stream_read_peek()). The function was a problem, because it returned a pointer to the internal buffer, which is now a ring buffer with wrapping. The new function just copies the data into a buffer, and in some cases requires callers to dynamically allocate memory. (The most common case, demux_lavf.c, required a separate buffer allocation anyway due to FFmpeg "idiosyncrasies".) This is the bulk of the demuxer_* changes. I'm not happy with this. There still isn't a good reason why there should be a ring buffer, that is complex, and most of the time just wastes half of the available memory. Maybe another rewrite soon. It also contains bugs; you're an alpha tester now.
2019-11-06 20:36:02 +00:00
ta_free(probe);
demux_libarchive: don't allow probing to read unlimited data Instead, allow reading 2KB only. This seems to be sufficient for libarchive to recognize zip, 7z, rar, tar. Good enough. This is implemented by creating an in-memory stream with a copy of the file header. If libarchive succeeds opening this, the actual stream is opened. Allowing unlimited reading could break unseekable streams, such as playing from http servers with no range request support or pipes. Also, we try not to read too much data in the first probe pass. Some slow network streams like shoutcast services could make probing much slower if we allow it to read too much. In the second probing pass, actually allow 200KB.
2015-08-24 20:21:36 +00:00
if (!ok)
return -1;
mpa = mp_archive_new(demuxer->log, demuxer->stream, flags);
stream: libarchive wrapper for reading compressed archives This works similar to the existing .rar support, but uses libarchive. libarchive supports a number of formats, including zip and (most of) rar. Unfortunately, seeking does not work too well. Most libarchive readers do not support seeking, so it's emulated by skipping data until the target position. On backwards seek, the file is reopened. This works fine on a local machine (and if the file is not too large), but will perform not so well over network connection. This is disabled by default for now. One reason is that we try libarchive on every file we open, before trying libavformat, and I'm not sure if I trust libarchive that much yet. Another reason is that this breaks multivolume rar support. While libarchive supports seeking in rar, and (probably) supports multivolume archive, our support of libarchive (probably) does not. I don't care about multivolume rar, but vocal users do.
2015-08-16 22:55:26 +00:00
if (!mpa)
return -1;
struct playlist *pl = talloc_zero(demuxer, struct playlist);
demuxer->playlist = pl;
// make it load archive://
pl->disable_safety = true;
char *prefix = mp_url_escape(mpa, demuxer->stream->url, "~|");
char **files = NULL;
int num_files = 0;
libarchive: unify entry iteration between stream/demux layers No really good reason to duplicate this.
2016-07-18 10:44:56 +00:00
while (mp_archive_next_entry(mpa)) {
stream: libarchive wrapper for reading compressed archives This works similar to the existing .rar support, but uses libarchive. libarchive supports a number of formats, including zip and (most of) rar. Unfortunately, seeking does not work too well. Most libarchive readers do not support seeking, so it's emulated by skipping data until the target position. On backwards seek, the file is reopened. This works fine on a local machine (and if the file is not too large), but will perform not so well over network connection. This is disabled by default for now. One reason is that we try libarchive on every file we open, before trying libavformat, and I'm not sure if I trust libarchive that much yet. Another reason is that this breaks multivolume rar support. While libarchive supports seeking in rar, and (probably) supports multivolume archive, our support of libarchive (probably) does not. I don't care about multivolume rar, but vocal users do.
2015-08-16 22:55:26 +00:00
// stream_libarchive.c does the real work
libarchive: prefix entry names in archive URLs with '/' This has the advantage that playlists within the archive will work as expected, because demux_playlist will correctly join the archive base URL and entry name. Before this change, it could skip before the "|", resulting in a broken URL.
2019-12-20 07:35:08 +00:00
char *f = talloc_asprintf(mpa, "archive://%s|/%s", prefix,
libarchive: unify entry iteration between stream/demux layers No really good reason to duplicate this.
2016-07-18 10:44:56 +00:00
mpa->entry_filename);
stream: libarchive wrapper for reading compressed archives This works similar to the existing .rar support, but uses libarchive. libarchive supports a number of formats, including zip and (most of) rar. Unfortunately, seeking does not work too well. Most libarchive readers do not support seeking, so it's emulated by skipping data until the target position. On backwards seek, the file is reopened. This works fine on a local machine (and if the file is not too large), but will perform not so well over network connection. This is disabled by default for now. One reason is that we try libarchive on every file we open, before trying libavformat, and I'm not sure if I trust libarchive that much yet. Another reason is that this breaks multivolume rar support. While libarchive supports seeking in rar, and (probably) supports multivolume archive, our support of libarchive (probably) does not. I don't care about multivolume rar, but vocal users do.
2015-08-16 22:55:26 +00:00
MP_TARRAY_APPEND(mpa, files, num_files, f);
}
if (files)
qsort(files, num_files, sizeof(files[0]), cmp_filename);
for (int n = 0; n < num_files; n++)
playlist_add_file(pl, files[n]);
demuxer->filetype = "archive";
demuxer->fully_read = true;
mp_archive_free(mpa);
demux: change hack for closing subtitle files early Subtitles (and a few other file types, like playlists) are not streamed, but fully read on opening. This means keeping the file handle or network socket open is a waste of resources and could cause other weird behavior. This is why there's a hack to close them after opening. Change this hack to make the demuxer itself do this, which is less weird. (Until recently, demuxer->stream ownership was more complex, which is why it was done this way.) There is some evil shit due to a huge ownership/lifetime mess of various objects. Especially EDL (the currently only nested demuxer case) requires being careful about mp_cancel and passing down stream pointers. As one defensive programming measure, stop accessing the "stream" variable in open_given_type(), even where it would still work. This includes removing a redundant line of code, and removing the peak call, which should not be needed anymore, as the remaining demuxers do this mostly correctly.
2018-09-07 21:02:36 +00:00
demux_close_stream(demuxer);
stream: libarchive wrapper for reading compressed archives This works similar to the existing .rar support, but uses libarchive. libarchive supports a number of formats, including zip and (most of) rar. Unfortunately, seeking does not work too well. Most libarchive readers do not support seeking, so it's emulated by skipping data until the target position. On backwards seek, the file is reopened. This works fine on a local machine (and if the file is not too large), but will perform not so well over network connection. This is disabled by default for now. One reason is that we try libarchive on every file we open, before trying libavformat, and I'm not sure if I trust libarchive that much yet. Another reason is that this breaks multivolume rar support. While libarchive supports seeking in rar, and (probably) supports multivolume archive, our support of libarchive (probably) does not. I don't care about multivolume rar, but vocal users do.
2015-08-16 22:55:26 +00:00
return 0;
}
const struct demuxer_desc demuxer_desc_libarchive = {
.name = "libarchive",
.desc = "libarchive wrapper",
.open = open_file,
};