Slightly cleaner. We don't need to awkwardly backup "some" state on
backwards playback. Due to not resetting last_format, normal timeline
switches don't unconditionally trigger recomputing of certain image
parameters. Also probably doesn't reset framedrop parameters, although I
don't care about that part.
This could lead to nonsense when backward playback is involved. Better
reduce the possible interactions. Besides, it's better to fully reset
things on seeks in general.
The only exception is has_broken_packet_pts, which enables hr-seek if
everything looks good. It's intended to trigger at the second hr-seek or
so if the file is normal, and to disable it if the file is broken. It
tries to avoid enabling the hr-seek logic before it can know about
whether things are "good", so resetting it on seeks would obviously
never enable it. Document it as explicit exception.
And add simpler aliases for the modes.
I'm not sure how to name things, and the option list is in general full
of different conventions. Some names are shortened, some are explicit
and long.
I guess options that have a chance to be used normally (i.e. not obscure
tuning or debugging) should have a short and convenient names.
In this specific case, play-direction is like a mixture of both. It
should be either playback-direction or play-dir, not shorten one word
but not the other.
The convenience aliases are because I got sick of typing out "backward".
I guess "back" would also do it, but there's no proper antonym (and
maybe it's "wrong" in the strict sense of the word).
Together with the previous commit, this seems to make backward playback
work in files with vorbis and mp3 audio codecs.
For Vorbis (with libavcodec's decoder, didn't test libvorbis), the first
packet was just always completely discarded. This happened even though
we tell libavcodec that we do discarding of padding manually. It simply
happened inside the codec, not libavcodec's general initial padding
handling. In addition, the first output decoded frame seems to contain
partial data. (Unlike the opus decoder, it doesn't report any padding at
all.)
The Opus decoder (again libavcodec only tested) reports an initial
padding, but it appears to be too small, and it sounds right only with 2
packets discarded. So its status doesn't change.
I'm not sure why I need 2 frames for mp3, but with that value I had
success on the samples I tested.
Some audio codecs will discard or cut the first frames when starting
decoding. While some of that works through well-defined mechanisms (like
initial padding), it's in general very codec/decoder specific, and not
really predictable. In addition, libavcodec isn't very good with
reporting "dropped" frames (and our internal interface reflects this).
It seems our only chance to handle this is through timestamps.
In theory, it would be best to discard frames that have timestamps
before the "resume" position. But since video has reordered timestamps,
we'd need to put some effort into finding this position. The first video
packet doesn't necessarily contain this timestamp. (In theory, we could
just do this in the demuxer with some trivial additional work, and set
it on the packet's kf_seek_pts field. Although this field is supposed to
contain just this value, the field is considered demuxer-internal, and I
didn't want to make matters worse by reusing it for the interface to the
decoder. With some more effort and buffering, we could calculate this
value within the decoder, but fuck that.)
The approach chosen in this commit is setting the timestamp to NOPTS.
This will break in some obscure situations, but backward playback is a
pretty obscure feature to begin with, so I considered this a reasonable
implementation choice.
Before passing a preroll packet to the decoder, its timestamps are set
to NOPTS. Frames that are returned from the decoder and have the NOPTS
timestamp are considered preroll and are discarded. This happens only
during "preroll" mode (preroll_discard==true), so it doesn't affect
normal forward playback. It's disabled on the first packet with a
timestamp, so it can tolerate some crap even in backward playback mode.
We don't check the dts fields out of laziness (decoded audio frames
don't even have this field).
I considered using an approach using the EDL clipping infrastructure (as
mentioned in the last two paragraphs in the commit message of commit
" demux_lavf: implement bad hack for backward playback of wav"). This
didn't work, and I blamed timestamp rounding within mpv for it. But the
problem was actually due to Matroska-rounded timestamps. Since the audio
frame size isn't exactly aligned to 1ms, there will be an overlap (or
gap) in the timestamps. This overlap is much smaller than 1ms, since
it's just the sub-millisecond remainder part of the audio frame size.
This makes the timestamps discontinuous and unreliable for the purpose
we wanted to use it. We can't just smooth the timestamps in the demuxer
either.
Matroska has this weird concept of "lacing", which are really sub-blocks
packed into a larger actual block. They are demuxed as individual
packets, because that's what the decoder needs. Basically they're a
Matroska hack to reduce per-packet muxing overhead.
One problem is that these sub-blocks don't have timestamps. The
timestamps need to be created from the "default duration". If this
default duration isn't in the file header (or if we drop it when it has
a known broken value), the resulting packets won't have a timestamp.
This is an usual and weird situation, that may confuse the demuxer layer
(and backward playback in particular). Fix this by not setting the
keyframe flag for these.
This affects only audio packets. Subtitle lacing is explicitly not
supported (in theory would at least lack a way to specify durations),
and video won't usually use it for real video codecs (not every frame is
a "keyframe", timestamp reordering).
Clarify existing semantics for the --start/--end/--length options.
De-emphasize the difference between absolute and relative timestamps,
since they've not been different by default since mpv 0.14.
Document a bug, that also happens to work as a feature: if the option
value begins with spaces, the code for checking for relative timestamps
is inactive, and they're always considered absolute. The check is done
on the first character of the string - so even a negative timestamp will
be treated as absolute.)
Yes, this is useful in extremely rare situations, such as when you
really want send a specific timestamp (even a negative one) to the
demuxer.
Another shitty obscure feature that usually nobody notices.
Unsurprisingly, it doesn't go well with backward playback mode.
If you use --keep-open in forward playback mode, and seek past the end
of the file, it tries to seek to the very last frame. The demuxer will
seek to the last "keyframe" before the end (i.e. some frames to go in
most cases), and trying to hr-seek to the file duration often won't cut
it, so this requires some special code. The function at hand seeks
"close" to the end, and then stops hr-seek when the last frame us
encountered (simple enough and very effective).
In backward playback mode, start and end are reversed, and we need to
seek "close" to the start of the file instead. Simple enough to do, and
it works.
One problem is that command.c has some weird logic to make going beyond
the last chapter either end playback (--keep-open=no), or jump to the
last frame. Now this will jump to the first frame, which is weird, but
let's ignore this.
Another problem is that seeking before playback start position hits EOF
in backward playback mode, which is a demuxer bug, and has nothing to do
with this code. But it triggers this code, so seeking before the start
will show the "last" frame. (My description is a mess with directions.
Figure it out yourself.)
Obviously should seek back to the end of the file when it loops.
Also remove some minor code duplication around start times. This isn't
the correct solution by the way. Rather than hoping we know a reasonable
start/end time, this stuff should instruct the demuxer to seek to the
exact location. It'll work with 99% of all normal files, but add an
appropriate comment (that basically says the function is bullshit) to
get_start_time() anyway.
This changes the behavior of the --ab-loop-a/b options. In addition, it
makes it work with backward playback mode.
The most obvious change is that the both the A and B point need to be
set now before any looping happens. Unlike before, unset points don't
implicitly use the start or end of the file. I think the old behavior
was a feature that was explicitly added/wanted. Well, it's gone now.
This is because of 2 reasons:
1. I never liked this feature, and it always got in my way (as user).
2. It's inherently annoying with backward playback mode.
In backward playback mode, the user wants to set A/B in the wrong order.
The ab-loop command will first set A, then B, so if you use this command
during backward playback, A will be set to a higher timestamps than B.
If you switch back to forward playback mode, the loop would stop
working. I want the loop to just continue to work, and the chosen
solution conflicts with the removed feature.
The order issue above _could_ be fixed by also switching the AB-loop
user option values around on direction switch. But there are no other
instances of option changes magically affecting other options, and doing
this would probably lead to unexpected misery (dying from corner cases
and such).
Another solution is sorting the A/B points by timestamps after copying
them from the user options. Then A/B options set in backward mode will
work in forward mode. This is the chosen solution. If you sort the
points, you don't know anymore whether the unset point is supposed to
signify the end or the start of the file.
The AB-loop code is slightly better abstracted now, so it should be easy
to restore the removed feature. It would still require coming up with a
solution for backwards playback, though.
A minor change is that if one point is set and the other is unset, I'm
rendering both the chapter markers and the marker for the set point.
Why? I don't know. My test file had chapters, and I guess I decided this
looked better.
This commit also fixes some subtle and obvious issues that I already
forgot about when I wrote this commit message. It cleans up some minor
code duplication and nonsense too.
Regarding backward playback, the code uses an unsanitary mix of internal
("transformed") and user timestamps. So the play_dir variable appears
more than usual.
To mention one unfixed issue: if you set an AB-loop that is completely
past the end of the file, it will get stuck in an infinite seeking loop
once playback reaches the end of the file. Fixing this reliably seemed
annoying, so the fix is "just don't do this". It's not a hard freeze
anyway.
This code attempts to seek to the last frame by seeking close to the
end, and then decoding until the last frame has been reached. To do so
it sets hrseek_lastframe, which for video enables some logic to "catch"
this last frame, and completely ignores hrseek_pts. But audio still may
use hrseek_pts
I don't know if the original author (me) was thinking, if anything, when
setting this variable to 1e99, essentially a random, number. It's very
large, and a timestamp like this will never happen, so it does its job.
But it's random.
Use INFINITY instead. It will skip all audio samples in the audio code
correctly. This change doesn't fix anything, but it does get rid of the
random looking number.
The get_play_start_pts() function was supposed to return "rebased"
(relative to 0) timestamps. This was roundabout, because one of 2
callers just added the offset back, and the other caller actually
expected an absolute timestamp.
Change rel_time_to_abs() (whose return value get_play_start_pts()
returns without further changes) to return absolute times.
This should fix that absolute and relative times passed to --start and
--end were treated the same, which can't be right. It probably also
fixes --end if --rebase-start-time=no is used (which can't have been
correct either).
All in all I'm not sure why --rebase-start-time=no or absolute vs.
relative times in --start/--end even exist, when they were incorrectly
implemented for years.
Untested, because no sample file and I don't care. However, if anyone
cares, and I got it wrong, I hope it's simple to fix.
Has been deprecated for almost 3 years. Manpage didn't mention the
deprecation, but CLI and release notes did. It wouldn't be much effort
to keep this option working, but I just don't see the damn point.
--start/--end can specify chapters using special syntax, which is
equivalent.
We need to transform the timestamp returned by get_play_end_pts().
I considered making it return the transformed timestamp directly. There
are 4 callers; 2 need a transformed timestamps, 2 don't. So I guess it
doesn't matter.
This adds the stops using the same logic get_play_end_pts() and
handle_loop_file(). It did that before, it just looks slightly different
now. It also won't try to add MP_NOPTS_VALUE as stop value.
Shitty ancient hack that wastes my time all the time.
demux.c: always return the coverart packet as soon as possible, and
don't let the backward demux state machine possibly stop it.
f_decoder_wrapper.c: mess with some shit until it somehow starts to
work. I think the old code tried to let it cleverly fall through so the
packet was processed "normally"; just make it run the "usual" code
instead.
This commit generally fixes backward playing in wav, at least in most
PCM cases.
libavformat's wav demuxer (and actually all other raw PCM based
demuxers) have a specific behavior that breaks backward demuxing. The
same thing also breaks persistent seek ranges in the demuxer cache,
although that's less critical (it just means some cached data gets
discarded). The backward demuxing issue is fatal, will log the message
"Demuxer not cooperating.", and then typically stop doing anything.
Unlike modern media formats, these formats don't organize media data in
packets, but just wrap a monolithic byte stream that is described by a
header. This is good enough for PCM, which uses fixed frames (a single
sample for all audio channels), and for which it would be too expensive
to have per frame headers.
libavformat (and mpv) is heavily packet based, and using a single packet
for each PCM frame causes too much overhead. So they typically "bundle"
multiple frames into a single packet. This packet size is obviously
arbitrary, and in libavformat's case hardcoded in its source code.
The problem is that seeking doesn't respect this arbitrary packet
boundary. Seeking is sample accurate. You can essentially seek inside a
packet. The resulting packets will not be aligned with previously
demuxed packets. This is normally OK.
Backward seeking (and some other demuxer layer features) expect that
demuxing an earlier demuxed file position eventually results in the same
packets, regardless of the seeks that were done to get there. I like to
call this "deterministic" demuxing. Backward demuxing in particular
requires this to avoid overlaps, which would make it rather hard to get
continuous output.
Fix this issue by detecting wav and hopefully other raw audio formats
with a heuristic (even PCM needs to be detected as heuristic). Then, if
a seek is requested, align the seek timestamps on the guessed number of
samples in the audio packets returned by the demuxer.
The heuristic excludes files with multiple streams. (Except "attachment"
video streams, which could be an ID3 tag. Yes, FFmpeg allows ID3 tags on
WAV files.) Such files will inherently use the packet concept in some
way.
We don't know how the demuxer chooses the internal packet size, but we
assume that it's fixed and aligned to PCM frame sizes. The frame size is
most likely given by block_align (the native wav frame size, according
to Microsoft). We possibly need to explicitly read and discard a packet
if the seek is done without reading anything before that. We ignore any
subsequent packet sizes; we need to avoid the very last packet, which
likely has a different size.
This hack should be rather benign. In the worst case, it will "round"
the seek target a little, but the maximum rounding amount is bounded.
Maybe we _could_ round up if SEEK_FORWARD is specified, but I didn't
bother.
An earlier commit fixed the same issue for mpv's demux_raw.
An alternative, and probably much better solution would be clipping
decoded data by timestamp. demux.c could allow the type of overlap the
wav demuxer introduces, and instruct the decoder to clip the output
against the last decoded timestamp. There's already an infrastructure
for this (demux_packet.end field) used by EDL/ordered chapters.
Although this sounds like a good solution, mpv unfortunately uses floats
for timestamps. The rounding errors break sample accuracy. Even if you
used integers, you'd need a timebase that is sample accurate (not always
easy, since EDL can merge tracks with different sample rates).
Yay, more subtle state on top of this nightmarish, fragile state
machine. But this is what happens when you subvert the laws of nature.
This simple checks where playback should "resume" from when no packets
were returned to the decoder yet after the seek that initiated backward
playback. The main purpose is to process the first returned keyframe
range in the same way like all other ranges. This ensures that things
like preroll are included properly.
Before this commit, it could for example have happened that the start of
the first audio frame was slightly broken, because no preroll was
included. Since the audio frame is reversed before sending it to the
audio output, it would have added an audible discontinuity before the
second frame was played; all subsequent frames would have been fine.
(Although I didn't test and confirm this particular issue.)
In future, this could be useful for certain other things.
At least the condition for delaying the backstep seek becomes simpler
and more explicit.
Move the code that attempts to start demuxing up in dequeue_packet.
Before, it was not called when the stream was in back_restarting state.
This commit makes streams be in back_restarting state at initialization,
so the demuxer would never have started reading.
Likewise, we need to call back_demux_see_packets() right after seek in
case the seek was within the cache. (We don't bother with checking
whether it was a cached seek; nothing happens if it was a normal one.)
There is nothing else that would process these cached packets
explicitly, although coincidences could sporadically trigger it.
The check for back_restart_next in find_backward_restart_pos() now
decides whether to use this EOF special code. Since the backward
playback start state also sets this variable, we don't need some of
the complex checks in dequeue_packet() anymore either.
As well as other filtering. I was writing this with the assumption that
timestamps go backwards (which I first planned to do). But in fact,
timestamps go forward, frame durations are positive, and adding a frame
duration to a timestamp yields the correct result. The only strange
thing is that timestamps are negative.
Also, media of course goes backwards. In other possible implementation,
filters would see normal forward playback, interrupted by seeks or
discontinuities. It turns out the current implementation of providing a
continuous backward media stream is probably better for filters.
Even deinterlacing seems to work. libavcodec always outputs fields in as
interleaved frames (i.e. fields are not reversed), and making up
timestamps for the new frames (when doubling the framerate) works
exactly like like in the forward case.
Actually the previous paragraph was a lie, and libavcodec does not
output fields as interleaved frames in rare cases. Sometimes AVFrame
contains single fields. In this case you'd need to inverse the field
dominance for deinterlacing filters to work correctly.
The way backward playback is implemented doesn't break basic assumptions
about timestamps after the decoder, so I guess all the encoding mode
needs to do is to adjust for the start offset, which it already does.
Though I might be wrong and my test was possibly flawed.
Stream recording on the other hand will fail immediately with
--record-file, and --stream-record will probably yield unexpected
results if any backstep seeks are done.
Make --audio-backward-overlap default to 2 for Opus. I have no idea why
this is needed. It seems to fix backward decoding though (going purely
by listening).
Normally, this should not be needed, since initial padding is completely
contained within the first packet (normally, and in the case I tested).
So the 2nd packet/frame should be fine, but for some unknown reason it
works only with the 3rd.
The only reasonable solution to this is probably to make discarding of
preroll frames based on timestmaps, instead of frame/packet count. But
then you get issues with video and its dumb timestamp reordering. So for
now, fuck it.
This seems more useful in general. This change also happens to fix a
miscounting of preroll packets when some of them were "rounded" away,
and which could make it stuck.
Also a simple intra-refresh encode with x264 (and muxed to mkv by it)
seems to work now. I guess I misinterpreted earlier results.
Backstepping still could get "stuck" if the demuxer didn't seek far back
enough. This commit fixes getting stuck if playing backwards from the
end, and audio has ended much earlier than the video.
In commit "demux: fix initial backward demuxing state in some cases",
I claimed that the backward seek semantics ("snapping" backward in
normal seeking, unrelated to backward playing) would take care of
this. Unfortunately, this is not always quite true.
In theory, a seek to any position (that does not use SEEK_FORWARD, i.e.
backward snapping) should return a packet for every stream. But I have a
mkv sample, where audio ends much earlier than video. Its mkvmerge
created index does not have entries for audio packets, so the video
index is used. This index has its last entry somewhere close after the
end of audio. So no audio packets will be returned. With a "too small"
back_seek_size, the demuxer will retry a seek target that ends up in
this place forever. (This does not happen if you use --index=recreate.
It also doesn't happen with libavformat, which always prefers its own
index, while mpv's internal mkv demuxer strictly prefers the index from
the file if it can be read.)
Fix this by adding the back_seek_size every time we fail to see enough
packets. This way the seek step can add up until it works.
To prevent that back_seek_pos just "runs away" towards negative infinity
by subtracting back_seek_size every time we back step to undo forward
reading (e.g. if --no-cache is used), readjust the back_seek_pos to the
lowest known resume position. (If the cache is active, kf_seek_pts can
be used, but to work in all situations, the code needs to grab the
minimum PTS in the keyframe range.)
Just rearranging shit. Setting SEEK_HR for backstep seeks actually
doesn't have much meaning, but disables the weird audio snapping for
"keyframe" seeks, and I don't know it's late.
This code used to be simpler, but now it's enough that it should be
factored into a single function.
Both uses of the new function are annoyingly different. The first use is
the special case when a decoder tries to read packets, but the demuxer
doesn't see any (like mp4 files with sparse video packets, which
actually turned out to be chapter thumbnail "tracks"). Then the other
stream queues will overflow, and the stream with no packets is marked
EOF to avoid stalling playback.
The second case is when the demxuer returns global EOF.
It would be more awkward to have the loop iterating the streams in the
function, because then you'd need a weird parameter to control the
behavior.
Just "mpv file.mkv --play-direction=backward" did not work, because
backward demuxing from the very end was not implemented. This is another
corner case, because the resume mechanism so far requires a packet
"position" (dts or pos) as reference. Now "EOF" is another possible
reference.
Also, the backstep mechanism could cause streams to find different
playback start positions, basically leading to random playback start
(instead of what you specified with --start). This happens only if
backstep seeks are involved (i.e. no cached data yet), but since this is
usually the case at playback start, it always happened. It was racy too,
because it depended on the order the decoders on other threads requested
new data. The comment below "resume_earlier" has some more blabla.
Some other details are changed.
I'm giving up on the "from_cache" parameter, and don't try to detect the
situation when the demuxer does not seek properly. Instead, always seek
back, hopefully some more.
Instead of trying to adjust the backstep seek target by a random value
of 1.0 seconds. Instead, always rely on the random value provided by the
user via --demuxer-backward-playback-step. If the demuxer should really
get "stuck" and somehow miss the seek target badly, or the user sets the
option value to 0, then the demuxer will not make any progress and just
eat CPU. (Although due to backward seek semantics used for backstep
seeks, even a very small seek step size will work. Just not 0.)
It seems this also fixes backstepping correctly when the initial seek
ended at the last keyframe range. (The explanation above was about the
case when it ends at EOF. These two cases are different. In the former,
you just need to step to the previous keyframe range, which was broken
because it didn't always react correctly to reaching EOF. In the latter,
you need to do a separate search for the last keyframe.)
Simple enough to do. May have mixed results. Typically, bitmap subtitles
will have a tight bounding box around the rendered text. But if for
example there is text on the top and bottom, it may be a single big
bitmap with a large transparent area between top and bottom. In
particular, DVD subtitles are really just a single screen-sized
RLE-encoded bitmap, though libavcodec will crop off transparent areas.
Like with sd_ass, you can't move subtitles _down_ if they are already in
their origin position. This could probably be improved, but I don't want
to deal with that right now.
Fixes the same thing as the previous commit did with demux_mkv. I'm not
sure if this is correct or a good idea (well, it works with my sample
file).
There are some shady things in this, but describing them would require
too many expletives.
In this scenario, the demuxer will output timestamps offset by the codec
delay (e.g. negative timestamps at the start; mkv simulates those), and
the trimming in the decoder (often libavcodec, but ad_lavc.c in our
case) will adjust the timestamps back (e.g. stream actually starts at
0).
This offset needs to be taken into account when seeking. This worked in
the uncached case. (demux_mkv.c is a bit tricky in that the index is
already in the offset space, so it compensates even though the seek call
does not reference codec_delay.) But in the cached case, seeks backwards
did not seek enough, and forward they seeked too much.
Fix this by adding the codec delay to the index search. We need to get
"earlier" packets, so e.g. seeking to position 0 really gets the initial
packets with negative timestamps.
This also adjusts the seek range start. This is also pretty obvious: if
the beginning of the file is cached, the seek range should start at 0,
not a negative value. We compare 0-based timestamps to it later on.
Not sure if this is the best approach. I also could have thought
about/checked some corner cases harder. But fuck this shit.
Not fixing duration (who cares) or end trimming, which would reduce the
seek range and duration (who cares).
This is a bad approach, and should be handled by a codec parameter field
(in mp_codec_params or AVCodecParameters).
It's bad because it's overly complicated, and has potential to break
demuxer cache assumptions: packets that were "intended" for seek
resuming may suddenly appear in the middle of a stream, when you seek
back and play a cached part again. (In general it was fine though,
because seek range joining tends to remove the first audio packet of the
next range when trying to find an overlap.)
demux_mkv.c does not try to export its codec_delay field through the
codec parameters mentioned above. In the only case I spotted this
element, the codec itself (opus) set this field within libavcodec. And I
think that's actually how it should be. On the other hand, a file could
in theory set this field via mkv headers if the codec is too stupid to
have such a field internally. But I don't really care until I see such a
file.
The end trimming is still sort of needed (though not sure if anything
uses it, other than the opus/mkv test sample I was using). The decoder
can't know whether something is the last packet, until it's too late.
The codec_delay field is still needed to offset timestamps.
Fixes stupid messages with a opus/mkv test file that had an absurdly
huge codec delay.
This file fully skips several frames at the start. ad_lavc.c trimmed
these frames to 0 samples and returned them. The next layer
(f_decoder_wrapper.c) saw discontinuous PTS values, because the PTS
values increased by a frame, but amounted to 0 audio samples. This was
harmless, but logged PTS discontinuity errors.
Not specifying a --start or using --start=100% with
--play-direction=backward usually does not work. The demuxer gets no
packets and immediately enters EOF state, which then hangs because
backward playback mode neither considers this mode, nor propagates the
EOF.
As far as demuxer implementations are concerned, this behavior is OK and
even wanted. Seeking near the end with SEEK_FORWARD set is allowed not
to return any packets (so a normal relative forward seek as done by the
user would end playback). Seeking exactly to the end or past it without
SEEK_FORWARD set is probably also sane.
Another vaguely related issue is that a backward seek during playback
start does not "establish" the demux position correctly: if stream A
hits the next keyframe and seeks back, while stream B has not had a
chance to read a packet yet, then stream B will never try to read from
the old position. The effect is that stream B (and thus playback) will
effectively miss the seek target. This is "random" because it depends on
the order and number of packet read calls made by the decoders.
Fixing this is probably hard, and requires extending the already complex
state machine with more states, so turn the manpage into a TODO list for
now.
Only timestamps that enter or leave the demuxer API should be adjusted
by ts_offset (which is usually the start time). queue_seek() is also
used by backward demux seeks, which uses an internal timestamp.
Raw audio formats can be accessed sample-wise, and logically audio
packets demuxed from it would contain only 1 sample. This is
inefficient, so raw audio demuxers typically "bundle" multiple samples
in one packet.
The problem for the demuxer cache and backward playback is that they
need properly aligned packets to make seeking "deterministic". The
requirement is that if you read some packets, and then seek back, you
eventually see the same packets again. demux_raw basically allowed to
seek into the middle of a previously returned packet, which makes it
impossible to make the transition seamless. (Unless you'd be aware of
the packet data format and cut them to make it seamless, which is too
complex for such a use case.)
Solve this by always aligning seeks to packet boundaries. This reduces
the seek accuracy to the arbitrarily chosen packet size. But you can use
hr-seek to fix this. The gain from not making raw audio an awful special
case pays in exchange for this "stupid" suggestion to use hr-seek.
It appears this also fixes that it could and did seek into the middle of
the frame (not sure if this code was ever tested - it goes back to
removing the code duplication between the former demux_rawaudio.c and
demux_rawvideo.c).
If you really cared, you could introduce a seek flag that controls
whether the seek is aligned or not. Then code which requires
"deterministic" demuxing could set it. But this isn't really useful for
us, and we'd always set the flag anyway, unless maybe the caching were
forced disabled.
libavformat's wav demuxer exhibits the same issue. We can't fix it (it
would require the unpleasant experience of contributing to FFmpeg), so
document this in otions.rst. In theory, this also affects seek range
joining, but the only bad effect should be that cached data is
discarded.
This is for uncompressed data, so every frame is a "keyframe". This is
part of making this demuxer work with the demuxer layer caching and
backward playback.
See manpage additions. This is a huge hack. You can bet there are shit
tons of bugs. It's literally forcing square pegs into round holes.
Hopefully, the manpage wall of text makes it clear enough that the whole
shit can easily crash and burn. (Although it shouldn't literally crash.
That would be a bug. It possibly _could_ start a fire by entering some
sort of endless loop, not a literal one, just something where it tries
to do work without making progress.)
(Some obvious bugs I simply ignored for this initial version, but
there's a number of potential bugs I can't even imagine. Normal playback
should remain completely unaffected, though.)
How this works is also described in the manpage. Basically, we demux in
reverse, then we decode in reverse, then we render in reverse.
The decoding part is the simplest: just reorder the decoder output. This
weirdly integrates with the timeline/ordered chapter code, which also
has special requirements on feeding the packets to the decoder in a
non-straightforward way (it doesn't conflict, although a bugmessmass
breaks correct slicing of segments, so EDL/ordered chapter playback is
broken in backward direction).
Backward demuxing is pretty involved. In theory, it could be much
easier: simply iterating the usual demuxer output backward. But this
just doesn't fit into our code, so there's a cthulhu nightmare of shit.
To be specific, each stream (audio, video) is reversed separately. At
least this means we can do backward playback within cached content (for
example, you could play backwards in a live stream; on that note, it
disables prefetching, which would lead to losing new live video, but
this could be avoided).
The fuckmess also meant that I didn't bother trying to support
subtitles. Subtitles are a problem because they're "sparse" streams.
They need to be "passively" demuxed: you don't try to read a subtitle
packet, you demux audio and video, and then look whether there was a
subtitle packet. This means to get subtitles for a time range, you need
to know that you demuxed video and audio over this range, which becomes
pretty messy when you demux audio and video backwards separately.
Backward display is the most weird (and potentially buggy) part. To
avoid that we need to touch a LOT of timing code, we negate all
timestamps. The basic idea is that due to the navigation, all
comparisons and subtractions of timestamps keep working, and you don't
need to touch every single of them to "reverse" them.
E.g.:
bool before = pts_a < pts_b;
would need to be:
bool before = forward
? pts_a < pts_b
: pts_a > pts_b;
or:
bool before = pts_a * dir < pts_b * dir;
or if you, as it's implemented now, just do this after decoding:
pts_a *= dir;
pts_b *= dir;
and then in the normal timing/renderer code:
bool before = pts_a < pts_b;
Consequently, we don't need many changes in the latter code. But some
assumptions inhererently true for forward playback may have been broken
anyway. What is mainly needed is fixing places where values are passed
between positive and negative "domains". For example, seeking and
timestamp user display always uses positive timestamps. The main mess is
that it's not obvious which domain a given variable should or does use.
Well, in my tests with a single file, it suddenly started to work when I
did this. I'm honestly surprised that it did, and that I didn't have to
change a single line in the timing code past decoder (just something
minor to make external/cached text subtitles display). I committed it
immediately while avoiding thinking about it. But there really likely
are subtle problems of all sorts.
As far as I'm aware, gstreamer also supports backward playback. When I
looked at this years ago, I couldn't find a way to actually try this,
and I didn't revisit it now. Back then I also read talk slides from the
person who implemented it, and I'm not sure if and which ideas I might
have taken from it. It's possible that the timestamp reversal is
inspired by it, but I didn't check. (I think it claimed that it could
avoid large changes by changing a sign?)
VapourSynth has some sort of reverse function, which provides a backward
view on a video. The function itself is trivial to implement, as
VapourSynth aims to provide random access to video by frame numbers (so
you just request decreasing frame numbers). From what I remember, it
wasn't exactly fluid, but it worked. It's implemented by creating an
index, and seeking to the target on demand, and a bunch of caching. mpv
could use it, but it would either require using VapourSynth as demuxer
and decoder for everything, or replacing the current file every time
something is supposed to be played backwards.
FFmpeg's libavfilter has reversal filters for audio and video. These
require buffering the entire media data of the file, and don't really
fit into mpv's architecture. It could be used by playing a libavfilter
graph that also demuxes, but that's like VapourSynth but worse.
The demuxer layer can start a thread to decouple the rest of the player
from blocking I/O (such as network accesses). But this particular
function does not support running with the thread enabled. The mutex use
within it is only since thread_work() may temporarily unlock the mutex,
and unlocking an unlocked mutex is not allowed. Most of the rest of the
code still does proper locking, even if it's pointless and effectively
single-threaded.
To make this look slightly cleaner, extend the mutex around the rest of
the code (like threaded code would have to do). This is mostly a
cosmetic change.
wm4