Commit Graph

643 Commits

Author SHA1 Message Date
Andreas Rheinhardt 3a822717bd avformat/matroskaenc: Check allocations implicit in dynamic buffers
Failures of the allocations that happen under the hood when using dynamic
buffers are usually completely unchecked and the Matroska muxer is no
exception to this.

The API has its part in this, because there is no documented way to
actually check for errors: The return value of both avio_get_dyn_buf()
as well as avio_close_dyn_buf() is only documented as "the length of
the byte buffer", so that using this to return errors would be an API
break.

Therefore this commit uses the only reliable way to check for errors
with avio_get_dyn_buf(): The AVIOContext's error flag. (This is one of
the advantages of avio_get_dyn_buf(): By not destroying the AVIOContext
it is possible to inspect this value.) Checking whether the size or the
pointer vanishes is not enough as it does not check for truncated output
(the dynamic buffer API is int based and so has to truncate the buffer
even when enough memory would be available; it's current actual limit is
even way below INT_MAX).

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-05-06 10:49:51 +02:00
Andreas Rheinhardt 8370c9c206 avformat/matroskaenc: Simplify writing buffer
If one already has the contents of a master elements in a buffer of
known size, then writing a EBML master element is no different from
writing an EBML binary element. It is overtly complicated to use
start/end_ebml_master() as these functions first write an unkown-length
size field of the appropriate length, then write the buffer's contents,
followed by a seek to the length field to overwrite it with the real
size (obtained via avio_tell() although it was already known in
advance), followed by another seek to the previous position. Just use
put_ebml_binary() instead.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-05-06 10:38:53 +02:00
Andreas Rheinhardt 3122dcf2fe avformat/matroskaenc: Avoid dynamic buffer when writing Colour
There is a good upper bound for the maximum length of the Colour master
element; it is therefore unnecessary to use a dynamic buffer for it.
A simple buffer on the stack is enough. This commit implements this.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-05-06 10:22:48 +02:00
Andreas Rheinhardt 851283a777 avformat/matroskaenc: Unify writing level 1 elements preliminarily
The Matroska muxer updates several header elements when the output is
seekable; if unseekable, the buffer containing the contents of the element
is immediately freed after writing. Before this commit, there were three
places doing exactly the same: Checking whether the output is seekable
and calling the function that writes and frees or the function that
just writes the EBML master. This has been unified; adding SeekHead
entries for these elements has been unified, too.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-05-06 09:45:17 +02:00
Andreas Rheinhardt 4ed3c925c3 avformat/matroskaenc: Move adding SeekEntry into end_ebml_master_crc32()
Up until now, SeekEntries were already added before
start_ebml_master_crc32() was even called and before we were actually
sure that we really write the element the SeekHead references: After
all, we might also error out later; and given that the allocations
implicit in dynamic buffers should be checked, end_ebml_master_crc32()
will eventually have to return errors itself, so that it is the right
place to add SeekHead entries.

The earlier behaviour is of course a remnant of the time in which
start_ebml_master_crc32() really did output something, so that the
position before start_ebml_master_crc32() needed to be recorded.
Erroring out later is also not as dangerous as it seems because in
this case no SeekHead will be written (if it happened when writing
the header, the whole muxing process would abort; if it happened
when writing the trailer (when writing chapters not available initially),
writing the trailer would be aborted and no SeekHead containing the
bogus chapter entry would be written).

This commit does not change the way the SeekEntries are added for those
elements that are output preliminarily; this is so because the SeekHead
is written before those elements are finally output and doing it
otherwise would increase the amount of seeks.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-05-06 09:27:40 +02:00
Andreas Rheinhardt 449eaeb7a7 avformat/matroskaenc: Check mimetypes earlier
This avoids errors lateron after the file header has already been
partially written.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-05-03 14:14:06 +02:00
Andreas Rheinhardt cb255b616c avformat/matroskaenc: Fix memleak upon encountering bogus chapter
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-05-03 14:05:44 +02:00
Andreas Rheinhardt 0aed3002ad avformat/matroskaenc: Make sure UIDs of delayed chapters are != 0
This has previously only been checked if the chapters were initially
available, but not if they were only written in the trailer.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-05-03 13:56:29 +02:00
Andreas Rheinhardt 6397b4d6a2 avformat/vorbiscomment: Switch to AVIOContext from bytestream API
Up until now ff_vorbiscomment_write() used the bytestream API to write
VorbisComments. Therefore the caller had to provide a sufficiently large
buffer to write the output.

Yet two of the three callers (namely the FLAC and the Matroska muxer)
actually want the output to be written via an AVIOContext; therefore
they allocated buffers of the right size just for this purpose (i.e.
they get freed immediately afterwards). Only the Ogg muxer actually
wants a buffer. But given that it is easy to wrap a buffer into an
AVIOContext this commit changes ff_vorbiscomment_write() to use an
AVIOContext for its output.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-05-03 13:43:54 +02:00
Andreas Rheinhardt 704d7c9f46 avformat/vorbiscomment: Replace AVDictionary ** by const AVDictionary *
ff_vorbiscomment_write() used an AVDictionary ** parameter for a
dictionary whose contents ought to be written; yet this can be replaced
by AVDictionary * since commit 042ca05f0fdc5f4d56a3e9b94bc9cd67bca9a4bc;
and this in turn can be replaced by const AVDictionary * to indicate
that the dictionary isn't modified; the latter also applies to
ff_vorbiscomment_length().

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-05-03 12:45:01 +02:00
Andreas Rheinhardt ca0a38f2f7 avformat/matroskaenc: Replace impossible condition with assert
If a FLAC track uses an unconventional channel layout, the Matroska
muxer adds a WAVEFORMATEXTENSIBLE_CHANNEL_MASK VorbisComment to the
CodecPrivate to preserve this information. And given that FLAC uses
24bit length fields, the muxer checks if the length is more than this
and errors out if it is.

Yet this can never happen, because we create the AVDictionary that is
the source for the VorbisComment. It only contains exactly one entry
that can't grow infinitely large (in fact, the length of the
VorbisComment is <= 4 + 33 + 1 + 18 + strlen(LIBAVFORMAT_IDENT)).
So we can simply assert the size to be < (1 << 24) - 4.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-05-03 12:44:54 +02:00
Andreas Rheinhardt d13feae0f8 avformat/matroskaenc: Write SeekHead when livestreaming
Commit 6fd300ac6c added support for WebM
Chunk livestreaming; in this case, both the header as well as each
Cluster is written to a file of its own, so that even if the AVIOContext
seems seekable, the muxer has to behave as if it were not. Yet one of
the added checks makes no sense: It ensures that no SeekHead is written
preliminarily (and hence no SeekHead is written at all) if the option
for livestreaming is set, although one should write the SeekHead in this
case when writing the Header. E.g. the WebM-DASH specification [1]
never forbids writing a SeekHead and in some instances (that don't apply
here) even requires it (if Cues are written after the Clusters).

[1]: https://sites.google.com/a/webmproject.org/wiki/adaptive-streaming/webm-dash-specification

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-05-03 12:44:48 +02:00
Andreas Rheinhardt 68c7186d94 avformat/matroskaenc: Only write Cluster if it has in fact been opened
Since commit 4aa0665f39, the dynamic
buffer destined for the contents of the current Cluster is no longer
constantly allocated, reallocated and then freed after writing the
content; instead it is reset and reused when closing a Cluster.

Yet the code in mkv_write_trailer() still checked for whether a Cluster
is open by checking whether the pointer to the dynamic buffer is NULL or
not (instead of checking whether the position of the current Cluster is
-1 or not). If a Cluster was not open, an empty Cluster would be output.

One usually does not run into this issue, because unless there are
errors, there are only three possibilities to not have an opened Cluster
at the end of writing a packet:
The first is if one sent an audio packet to the muxer. It might trigger
closing and outputting the old Cluster, but because the muxer caches
audio packets internally, it would not be output immediately and
therefore no new Cluster would be opened.
The second is an audio packet that does not contain data (such packets
are sometimes sent for side-data only, e.g. by the FLAC encoder). The
only difference to the first scenario is that such packets are not
cached.
The third is if one explicitly flushes the muxer by sending a NULL
packet via av_write_frame().

If one also allows for errors, then there is also another possibility:
Caching the audio packet may fail in the first scenario.

If one calls av_write_trailer() after the first scenario, the cached
audio packet will be output when writing the trailer, for which
a Cluster is opened and everything is fine; because flushing the muxer
does currently not output the cached audio packet (if one is cached),
the issue also does not exist if an audio packet has been cached before
flushing. The issue only exists in one of the other scenarios.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-05-03 12:44:44 +02:00
Andreas Rheinhardt 904b25a550 avformat/matroskaenc: Use comparison instead of assignment
This bug was introduced in 3589b3f2e2.
Fixes Coverity ID 1462425.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-22 03:45:44 +02:00
Andreas Rheinhardt 8a4fda029a avformat/matroskaenc: Cosmetics
Reindentation, removal of { } if they contain only one statement
and moving the return statement to a line of its own in situations
like "if (ret < 0) return ret;". Moreover, several overlong lines
were made shorter and a camelCase variable received a name in line
with our naming conventions.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-21 08:50:22 +02:00
Andreas Rheinhardt e3c54b549a avformat/matroskaenc: Redo handling of FlagDefault
Up until now, the Matroska muxer would mark a track as default if it had
the disposition AV_DISPOSITION_DEFAULT or if there was no track with
AV_DISPOSITION_DEFAULT set; in the latter case even more than one track
of a kind (audio, video, subtitles) was marked as default which is not
sensible.

This commit changes the logic used to mark tracks as default. There are
now three modes for this:
a) In the "infer" mode the first track of every type (audio, video,
subtitles) with default disposition set will be marked as default; if
there is no such track (for a given type), then the first track of this
type (if existing) will be marked as default. This behaviour is inspired
by mkvmerge. It ensures that the default flags will be set in a sensible
way even if the input comes from containers that lack the concept of
default flags. This mode is the default mode.
b) The "infer_no_subs" mode is similar to the "infer" mode; the
difference is that if no subtitle track with default disposition exists,
no subtitle track will be marked as default at all.
c) The "passthrough" mode: Here the track will be marked as default if
and only the corresponding input stream had disposition default.

This fixes ticket #8173 (the passthrough mode is ideal for this) as
well as ticket #8416 (the "infer_no_subs" mode leads to the desired
output).

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-21 08:36:20 +02:00
Andreas Rheinhardt 75e50c3141 avformat/matroskaenc: Don't needlessly copy AVCodecParameters
At the end of encoding, the FLAC encoder sends a packet whose side data
contains updated extradata (e.g. a correct md5 checksum). The Matroska
muxer uses this to update the CodecPrivate.

In doing so, the stream's codecpar was copied. But given that writing
a FLAC CodecPrivate does not modify the used AVCodecParameters at all,
there is no need to do so and this commit changes this.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-21 08:28:58 +02:00
Andreas Rheinhardt abc7dc32bd avformat/matroskaenc: Add const where appropriate
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-21 08:20:02 +02:00
Andreas Rheinhardt 5b6e164b4b avformat/matroskaenc: Don't waste bytes on length fields
Several EBML Master elements for which a good upper bound of the final
length was available were nevertheless written without giving an
upper bound of the final length to start_ebml_master(), so that their
length fields were eight bytes long. This has been changed.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-21 08:11:09 +02:00
Andreas Rheinhardt a9844341f7 avformat/matroskaenc: Only write Tracks if there is a track
The Matroska muxer does not write every stream as a Matroska track;
some streams are written as AttachedFile. But should no stream be
written as a Matroska track, the Matroska muxer would nevertheless
write a Tracks element without a TrackEntry. This is against the spec.
This commit changes this and only writes a Tracks if there is a Matroska
track.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-21 07:49:39 +02:00
Andreas Rheinhardt ef45cccc81 avformat/matroskaenc: Warn that WebM doesn't support Attachments
As WebM doesn't support Attachments, the Matroska muxer drops them when
in WebM mode. This happened silently until this commit which adds a
warning for this.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-21 07:41:21 +02:00
Andreas Rheinhardt 4dd63ae86c avformat/matroskaenc: Don't use size of inexistent Cluster
In order to determine whether the current Cluster needs to be closed
because of the limits on clustersize and clustertime,
mkv_write_packet() would first get the size of the current Cluster by
applying avio_tell() on the dynamic buffer holding the current Cluster.
It did this without checking whether there is a dynamic buffer for
writing Clusters open right now.

In this case (which happens when writing the first packet)
avio_tell() returned AVERROR(EINVAL); yet it is not good to rely on
avio_tell() (or actually, avio_seek()) to handle the situation
gracefully.

Fixing this is easy: Only check whether a Cluster needs to be closed
if a Cluster is in fact open.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-21 07:33:04 +02:00
Andreas Rheinhardt 86de864741 avformat/matroskaenc: Add check for using explicit TrackNumber
When creating DASH streams, the TrackNumber is externally prescribed
and not derived from the number of streams in the AVFormatContext, so
if the number of tracks for a file using an explicit TrackNumber was
more than one, the resulting file would be broken (it would be impossible
to tell to which track a Block belongs if different tracks share the
same TrackNumber). So disallow this.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-21 07:31:10 +02:00
Andreas Rheinhardt 945b928730 avformat/matroskaenc: Improve Cues in case of no video
The Matroska muxer currently only adds CuePoints in three cases:
a) For video keyframes. b) For the first audio frame in a new Cluster if
in DASH-mode. c) For subtitles. This means that ordinary Matroska audio
files won't have any Cues which impedes seeking.

This commit changes this. For every track in a file without video track
it is checked and tracked whether a Cue entry has already been added
for said track for the current Cluster. This is used to add a Cue entry
for each first packet of each track in each Cluster.

Implements #3149.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-20 21:30:46 +02:00
Andreas Rheinhardt 13c12cd470 avformat/matroskaenc: Remove limit on the number of tracks
The Matroska file format has practically no limit on the number of
tracks (the current limit is 2^56 - 1); yet because they are encoded in
a variable length format in (Simple)Blocks this muxer has simply imposed
a limit on the number of tracks in order to ensure that they can always
be written on one byte in order to simplify the muxing process.

This commit removes said limit.

Also, zero is an invalid TrackNumber, so disallow this value in the
dash_track_number option.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-20 21:30:46 +02:00
Andreas Rheinhardt 112afaccdf avformat/matroskaenc: Refactor writing EBML lengths
This commit factors the ability to write ordinary EBML numbers out of
the functions for writing EBML lengths. This is in preparation for
future commits.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-20 21:30:46 +02:00
Andreas Rheinhardt 40d038a635 avformat/matroskaenc: Rename functions to better reflect what they do
EBML uses variable length integers both for the EBML IDs as well as for
the EBML lengths; Matroska also uses them for the TrackNumber in
(Simple)Blocks and for the lengths of laces when EBML lacing is used.

When encoding EBML lengths, certain encodings have a special meaning,
namely that the element has an unknown length. This is not so when
encoding general EBML variable length integers.

Yet the functions called ebml_num_size() and put_ebml_num() had this
special meaning hardcoded, i.e. they are there to write EBML lengths and
not general EBML numbers. So rename them.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-20 21:30:46 +02:00
Andreas Rheinhardt 9b0f9003df avformat/matroskaenc: Make ebml_num_size() more robust
Matroska (or actually EBML) uses variable-length numbers where only
seven bits of every byte is usable for the length; the other bits encode
the length of the variable-length number. So in order to find out how
many bytes one needs to encode a given number one can use a loop like
while (num >> 7 * bytes) bytes++; the Matroska muxer effectively did this.

Yet it has a disadvantage: It is impossible for the result of a single
right shift of an unsigned number with most significant bit set to be
zero, because one can only shift by 0..(width - 1). On some
architectures like x64 it is not even possible to do it with undefined
right shifts in which case this leads to an infinite loop.

This can be easily avoided by switching to a loop whose condition is
(num >>= 7). The maximum value the so modified function can return
is 10; any value > 8 is invalid and will now lead to an assert in
put_ebml_num() or in start_ebml_master() (or actually in
put_ebml_size_unknown()).

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-20 21:30:46 +02:00
Andreas Rheinhardt 3589b3f2e2 avformat/matroskaenc: Improve mimetype search
Use the mime_types of the corresponding AVCodecDescriptor instead of
tables specific to Matroska. The former are generally more encompassing:
They contain every item of the current lists except "text/plain" for
AV_CODEC_ID_TEXT and "binary" for AV_CODEC_ID_BIN_DATA.

The former has been preserved by special-casing it while the latter is
a hack added in c9212abf so that the demuxer (which uses the same tables)
sets the appropriate CodecID for broken files ("binary" is not a correct
mime type at all); using it for the muxer was a mistake. The correct
mime type for AV_CODEC_ID_BIN_DATA is "application/octet-stream" and
this is what one gets from the AVCodecDescriptor.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-20 21:00:19 +02:00
Andreas Rheinhardt f0d712d0f9 avformat/matroskaenc: Don't write elements with their default value
This has happened when writing chapters: Both editions as well as
chapters are by default not hidden and given that we don't support
writing hidden chapters at all, we don't need to write said elements at
all. The same goes for ChapterFlagEnabled.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-14 03:55:06 +02:00
Andreas Rheinhardt 59e3a9aede avformat/matroskaenc: Change signature of mkv_write_track()
Up until now, mkv_write_track() received the index of the stream whose
header data it is about to write as parameter; this index has until
recently been explicitly used to generate both TrackNumber and TrackUID.
But this is no longer so and as there is no reason why the function
for writing a single TrackEntry should even know the index of the
TrackEntry it is about to write, said index is replaced in the list of
function parameters by the corresponding AVStream and mkv_track.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-13 09:09:25 +02:00
Andreas Rheinhardt 385964409a avformat/matroskaenc: Automatically use right TrackNumber in Cues
mkv_cuepoint (the structure used to store the index entries in the
Matroska muxer) currently contains fields for both the index of the
packet's stream in the AVFormatContext.streams array and for the
Matroska TrackNumber; correspondingly, mkv_add_cuepoint() has parameters
for both. But these two numbers can't be chosen independently, so get
rid of the TrackNumber.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-13 09:08:41 +02:00
Andreas Rheinhardt 629f08a863 avformat/matroskaenc: Ignore AttachedFiles for track limit
Attachments are streams in FFmpeg, but they are not tracks in Matroska.
Yet they were counted when checking a limit for the number of tracks that
the Matroska muxer imposes. This is unnecessary and has been changed.

Also use unsigned variables for the variables denoting TrackNumbers as
negative TrackNumbers are impossible.

(The Matroska file format actually has practically no limit on the
number of tracks and this is purely what our muxer supports. But even if
this limit were removed/relaxed in the future, it still makes sense to
use small TrackNumbers as this patch does, because greater numbers need
more bytes to encode.)

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-13 08:51:21 +02:00
Andreas Rheinhardt ccadd00a4a avformat/matroskaenc: Make output more deterministic
Using random values for TrackUID and FileUID (as happens when the
AVFMT_FLAG_BITEXACT flag is not set) has the obvious downside of making
the output indeterministic. This commit mitigates this by writing the
potentially random values with a fixed size of eight byte, even if their
actual values would fit into less than eight bytes. This ensures that
even in non-bitexact mode, the differences between two files generated
with the same settings are restricted to a few bytes in the header.
(Namely the SegmentUID, the TrackUIDs (in Tracks as well as when
referencing them via TagTrackUID), the FileUIDs (in Attachments as
well as in TagAttachmentUID) as well as the CRC-32 checksums of the
Info, Tracks, Attachments and Tags level-1-elements.) Without this
patch, there might be an offset/a size difference between two such
files.

The FATE-tests had to be updated because the fixed-sized UIDs are also
used in bitexact mode.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-13 08:43:14 +02:00
Andreas Rheinhardt 358b58cb29 avformat/matroskaenc: Remove allocations for Attachments
If there are Attachments to write, the Matroska muxer currently
allocates two objects: An array that contains an entry for each
AttachedFile containing just the stream index of the corresponding
stream and the FileUID used for this AttachedFile; and a structure with
a pointer to said array and a counter for said array. These uids are
generated via code special to Attachments: It uses an AVLFG in the
normal and a sha of the attachment data in the bitexact case. (Said sha
requires an allocation, too.)

But now that an uid is generated for each stream in mkv_init(), there is
no need any more to use special code for generating the FileUIDs of
AttachedFiles: One can simply use the uid already generated for the
corresponding stream. And this makes the whole allocations of the
structures for AttachedFiles as well as the structures itself superfluous.
They have been removed.

In case AVFMT_FLAG_BITEXACT is set, the uids will be different from the
old ones which is the reason why the FATE-test lavf-mkv_attachment
needed to be updated. The old method had the drawback that two
AttachedFiles with the same data would have the same FileUID.
The new one doesn't.

Also notice that the dynamic buffer used to write the Attachments leaks
if an error happens when writing the buffer. By removing the
allocations potential sources of errors have been removed.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-13 08:34:59 +02:00
Andreas Rheinhardt 4b18999bee avformat/matroskaenc: Reuse random seed
This commit reuses the random seed generated in mkv_init() (to determine
the TrackUIDs) for the SegmentUID in order to avoid a potentially
expensive call to av_get_random_seed().

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-13 08:27:19 +02:00
Andreas Rheinhardt 45314ff21a avformat/matroskaenc: Use random TrackUID
Up until now, the TrackUID of a Matroska track which is supposed to be
random was not random at all: It always coincided with the TrackNumber
which is usually the 1-based index of the corresponding stream in the
array of AVStreams. This has been changed: It is now set via an AVLFG
if AVFMT_FLAG_BITEXACT is not set. Otherwise it is set like it is set
now (the only change happens if an explicit track number has been
chosen via dash_track_number, because the system used in the normal
situation is now used, too). In particular, no FATE tests need to be
updated.

This also fixes a bug in case the dash_track_number option was used:
In this case the TrackUID was set to the provided number, but the tags
were written with a TagTrackUID simply based upon the index, so that
the tags didn't apply to the track they ought to apply to.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-13 08:17:00 +02:00
Andreas Rheinhardt bd7dea3f4f avformat/matroskaenc: Don't waste bytes writing durations
Tags in the Matroska file format can be summarized as follows: There is
a level 1-element called Tags containing one or many Tag elements each
of which in turn contain a Targets element and one or many SimpleTags.
Each SimpleTag roughly corresponds to a single key-value pair similar to
an AVDictionaryEntry. The Targets meanwhile contains information to what
the metadata contained in the SimpleTags contained in the containing Tag
applies (i.e. to the file as a whole or to an individual track).

The Matroska muxer writes such metadata. It puts the metadata of every
stream into a Tag whose Targets makes it point to the corresponding
track. And if the output is seekable, then it also adds another Tag for
each track whose Targets corresponds to the track and where it reserves
space in a SimpleTag to write the duration at the end of the muxing
process into.

Yet there is no reason to write two Tag elements for a track and a few
bytes (typically 24 bytes per track) can be saved by adding the duration
SimpleTag to the other Tag of the same track (if it exists).

FATE has been updated because the output files changed. (Tests that
write to unseekable output (pipes) needn't be updated (no duration tag
has ever been written for them) and the same applies to tests without
further metadata.)

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-13 08:08:42 +02:00
Andreas Rheinhardt 3c3ad1deb0 avformat/matroskaenc: Ensure that ChapterUID are != 0
AVChapters have an int as id field and therefore this value can appear
<= 0. When remuxing from Matroska, this value actually contains
the lower 32 bits of the original ChapterUID (which can be 64 bits).

In order to ensure that the ChapterUID is always > 0, they were offset
as follows (since 07704c61): First max(0, 1LL - chapter[i].id) was computed
and stored in an uint32_t. And then the IDs were offset using this value.

This has two downsides:
1. It does not ensure that the UID is actually != 0: Namely if there is
a chapter with id == INT_MIN, then the offset will be 2^31 + 1 and a
chapter with id == INT_MAX will become 2^31 - 1 + 2^31 + 1 = 2^32 = 0,
because the actual calculation was performed in 32 bits.
2. As soon as a chapter id appears to be negative, a nontrivial offset
is used, so that not even a ChapterUID that only uses 32 bits is
preserved.

So change this by treating the id as an unsigned value internally and
only offset (by 1) if an id vanishes. The actual offsetting then has to
be performed in 64 bits in order to make sure that no UINT32_MAX wraps
around.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-13 07:59:23 +02:00
Andreas Rheinhardt 081ef58619 avformat/matroskaenc: Remove unnecessary headers
subtitles.h has been included in order to use ff_subtitles_next_line()
to help parsing srt subtitles which at that time had their timing as
part of the payload and not as part of the AVPacket fields. When this
changed (in 55180b32) it has been forgotten to remove this header.

libavcodec/internal.h meanwhile has been added in bb47aa5850 and has
never been used at all.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-10 00:32:42 +02:00
Andreas Rheinhardt 52523b6963 avformat/matroskaenc: Improve BlockAdditions
8ffcc826 added support for muxing BlockAdditions with BlockAddID equal
to one. The restriction to BlockAddID == 1 probably resulted from
a limitation to what was needed; yet over time this led to three
occurences of "(side_data_size && additional_id == 1)". This commit
changes this by setting side_data_size to 0 if additional_id != 1.

It also stops hardcoding 1 for the value of BlockAddID to write;
but it still upholds the requirement that it is 1. See below.

Despite BlockAddId actually having a default value of 1, it is still
written, because until very recently (namely dbc50f8a) our demuxer
used a wrong default value of 0.

Furthermore, use put_ebml_binary() to write the BlockAdditional element.

(The Matroska specifications have evolved and now the BlockAddID 1 is
reserved for the codec (as described in the codec's codec mapping),
BlockMore elements with BlockAddID > 1 are now of a more
codec-independent nature and require a BlockAdditionalMapping in the
track's TrackEntry. Given that this muxer does not support writing said
BlockAdditionalMapping yet (actually, none have been defined yet), we
have to uphold the requirement that BlockAddID == 1.)

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-03 09:04:22 +02:00
Andreas Rheinhardt af97a3a4d6 avformat/matroskaenc: Improve checks for updating Tags
When updating the Tags at the end, the Matroska muxer would twice check
for whether (!mkv->is_live) is true, despite this code being only executed
if it is. Furthermore, a loop iterates over all the streams even when
there is no Tags element to update at all, because the check for whether
there are Tags is only performed later. This commit fixes this.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-03 08:22:04 +02:00
Andreas Rheinhardt 0fc150f048 avformat/matroskaenc: Remove unnecessary avio_tell(), avio_seek()
avio_close_dyn_buf() has a bug: When the write pointer does not point to
the end of the written data when calling it (i.e. when one has performed
a seek back to update already written data), it would not add padding to
the end of the buffer, but to the current position, overwriting other
data; furthermore the reported size would be wrong (off by the amount of
data it has overwritten with padding).

In order not to run into this when updating already written elements or
elements for which size has only been reserved, the Matroska muxer would
first record the current position of the dynamic buffer, then seek to
the desired position, perform the update and seek back to the earlier
position.

But now that end_ebml_master_crc32() does not make use of
avio_close_dyn_buf() any more, this is no longer necessary.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-03 08:10:57 +02:00
Andreas Rheinhardt 4aa0665f39 avformat/matroskaenc: Stop reallocating of Cluster buffer
The Matroska muxer uses a dynamic buffer to buffer the content of
Clusters before eventually writing them. Up until now, each time a
Cluster was written, the dynamic buffer was closed, i.e. freed; now it
is only reset, saving allocations of the AVIOContext itself, its opaque
as well as most of the reallocations of the buffer.

This is advantageous performance-wise, in particular on systems where
reallocations are slow (namely Windows). The following table shows the
decicyles for writing a frame on Linux (Ubuntu 19.10) and Windows (7)
on an x64 Haswell (to /dev/null on Linux, to stdout which is discarded
on Windows (the default values of the size and duration of clusters for
seekable output have been explicitly set in this case); in all tests,
writing CRC-32 values has been disabled in all tests; calls to the muxer's
write_packet function in write_packet() in libavformat/mux.c have been
timed; each of the following tests has been repeated 50 times):

    | Windows before | Windows after | Linux before | Linux after
_________________________________________________________________
 A  |     979437     |    192304     |    259500    |   183320
 B  |     715936     |    155648     |    152786    |   130879
 C  |     265115     |     56034     |     78496    |    53243
 D  |     386224     |     80307     |    128894    |    75354
 E  |      21732     |     10695     |     11320    |     9801

(A is a 10.2 mb/s file with a GOP length of 2s, amounting to an average
Cluster size of about 2.5 MiB; the average Cluster size of B is 1.1 MiB;
for C it is 2.35 MiB, for D it is 0.46 MiB; for E - a file with just a
single audio track of 158kb/s resulting in a Cluster size of about 100
kB, the relative gains were the smallest, probably because of the small
Cluster size.)

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-03 07:53:25 +02:00
Andreas Rheinhardt da4ba2431c avformat/matroskaenc: Don't implicitly mark WebVTT in WebM as English
Writing the language of WebVTT in WebM proceeded differently than the
language of all other tracks: In case no language was given, it does not
write anything instead of "und" (for undefined). Because the default
value of the Language element in WebM (that inherited it from Matroska)
is "eng" (for English), any such track will actually be flagged as
English.

Doing it this way goes back to commit 509642b4 (the commit adding
support for WebVTT) and no reason for this has been given in the commit
message or in the discussion about this patch on the mailing list; the
best I can think of is this: the WebM wiki contains "The srclang attribute
is stored as the Language sub-element." Someone unfamiliar with default
values in Matroska/WebM could interpret this as meaning that no Language
element should be written if the language is unknown. And this is wrong
and this commit changes it.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-02 20:40:44 +02:00
Andreas Rheinhardt bc52ce309d avformat/matroskaenc: Reindent after previous commit
Also remove { } after an if if there is only one statement inside { }.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-02 20:38:07 +02:00
Andreas Rheinhardt 0d4b3b4c02 avformat/matroskaenc: Combine checks for audio
mkv_write_track() currently has three places where it checks for whether
the current codec type is audio: One in a switch and two outside of it.
These checks can be combined by moving the code after the other two checks
inside the audio-related part of the switch.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-02 20:23:20 +02:00
Andreas Rheinhardt 98a6c6ec82 avformat/matroskaenc: Simplify writing Void elements
Reserving space in Matroska works by writing a Void element. And until
now this worked as follows: The current position was recorded and the
EBML ID as well as the length field written; then the new position was
recorded to know how much more to write. Afterwards the actual writing
has been performed via ffio_fill().

But it is unnecessary to explicitly use the positions (obtained via
avio_tell()) to find out how much still needs to be written, because the
length of the ID and the length field are known. So rewrite the function
to no longer use them.

Also, given that ffio_fill() uses an int parameter and given that no
current caller (and no sane future caller) will want to reserve several
GB of space, make the size parameter of put_ebml_void() itself an int.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-02 20:12:00 +02:00
Andreas Rheinhardt 8e4c871196 avformat/matroskaenc: Avoid seek when writing Cues at the front
When the Cues are written in front of the Cluster, the muxer would seek
to the beginning (to where the Cues ought to be written) and write the
Cues; afterwards it would seek back to the end of the file only to seek
to the beginning once again to update several elements there. This
commit removes the seek to the end.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-02 20:00:34 +02:00
Andreas Rheinhardt b788343446 avformat/matroskaenc: Fix edge case of writing Cues at the beginning
The Matroska muxer has the ability to write the Cues (the index) at the
beginning of the file (in front of the Cluster): The user inputs the
amount of space that should be reserved at the beginning of the file and
if this is sufficient, the Cues will be written there and the part of the
reserved space not used up by the Cues will be filled with a "Void"
element.

There is just one problem with this: One can not fill a single byte this
way, because said Void element is minimally two bytes long (one byte ID,
one byte length field). Up until now, if one reserved one byte more than
needed, one would run into an assert when writing the Void element.

There are two solutions for this: Error out if it happens. Or adjust the
length field of the Cues in order to ensure that the above situation
can't happen (i.e. write the length on one byte more than necessary).
The first solution is very unsatisfactory, as enough space has been
reserved. Therefore this commit implements the second solution.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-04-02 19:50:05 +02:00