Decoding H264 using Video Decode Acceleration used the custom 'vda_h264_dec'
decoder in FFmpeg.
The Good: This new implementation has some advantages over the previous one:
- It works with Libav: vda_h264_dec never got into Libav since they prefer
client applications to use the hwaccel API.
- It is way more efficient: in my tests this implementation yields a
reduction of CPU usage of roughly ~50% compared to using `vda_h264_dec` and
~65-75% compared to h264 software decoding. This is mainly because
`vo_corevideo` was adapted to perform direct rendering of the
`CVPixelBufferRefs` created by the Video Decode Acceleration API Framework.
The Bad:
- `vo_corevideo` is required to use VDA decoding acceleration.
- only works with versions of ffmpeg/libav new enough (needs reference
refcounting). That is FFmpeg 2.0+ and Libav's git master currently.
The Ugly: VDA was hardcoded to use UYVY (2vuy) for the uploaded video texture.
One one end this makes the code simple since Apple's OpenGL implementation
actually supports this out of the box. It would be nice to support other
output image formats and choose the best format depending on the input, or at
least making it configurable. My tests indicate that CPU usage actually
increases with a 420p IMGFMT output which is not what I would have expected.
NOTE: There is a small memory leak with old versions of FFmpeg and with Libav
since the CVPixelBufferRef is not automatically released when the AVFrame is
deallocated. This can cause leaks inside libavcodec for decoded frames that
are discarded before mpv wraps them inside a refcounted mp_image (this only
happens on seeks).
For frames that enter mpv's refcounting facilities, this is not a problem
since we rewrap the CVPixelBufferRef in our mp_image that properly forwards
CVPixelBufferRetain/CvPixelBufferRelease calls to the underying
CVPixelBufferRef.
So, for FFmpeg use something more recent than `b3d63995` for Libav the patch
was posted to the dev ML in July and in review since, apparently, the proposed
fix is rather hacky.
This is based on the MPlayer VA API patches. To be exact it's based on
a very stripped down version of commit f1ad459a263f8537f6c from
git://gitorious.org/vaapi/mplayer.git.
This doesn't contain useless things like benchmarking hacks and the
demo code for GLX interop. Also, unlike in the original patch, decoding
and video output are split into separate source files (the separation
between decoding and display also makes pixel format hacks unnecessary).
On the other hand, some features not present in the original patch were
added, like screenshot support.
VA API is rather bad for actual video output. Dealing with older libva
versions or the completely broken vdpau backend doesn't help. OSD is
low quality and should be rather slow. In some cases, only either OSD
or subtitles can be shown at the same time (because OSD is drawn first,
OSD is prefered).
Also, libva can't decide whether it accepts straight or premultiplied
alpha for OSD sub-pictures: the vdpau backend seems to assume
premultiplied, while a native vaapi driver uses straight. So I picked
straight alpha. It doesn't matter much, because the blending code for
straight alpha I added to img_convert.c is probably buggy, and ASS
subtitles might be blended incorrectly.
Really good video output with VA API would probably use OpenGL and the
GL interop features, but at this point you might just use vo_opengl.
(Patches for making HW decoding with vo_opengl have a chance of being
accepted.)
Despite these issues, decoding seems to work ok. I still got tearing
on the Intel system I tested (Intel(R) Core(TM) i3-2350M). It was also
tested with the vdpau vaapi wrapper on a nvidia system; however this
was rather broken. (Fortunately, there is no reason to use mpv's VAAPI
support over native VDPAU.)
Instead of generating vdpau_template.c with a Perl script, just include
the generated file in git. This is ok because it changes very rarely,
and the script is larger than the output it generates.
It also simplify the Makefile, and fixes the build. The problem was that
transitive dependencies do not work with generated files: there is no
dependency information yet when building it the first time. I overlooked
this because I didn't delete the .d files for testing (which contained
the correct dependencies, but only _after_ a first successful build).
Move the decoder parts from vo_vdpau.c to a new file vdpau_old.c. This
file is named so because because it's written against the "old"
libavcodec vdpau pseudo-decoder (e.g. "h264_vdpau").
Add support for the "new" libavcodec vdpau support. This was recently
added and replaces the "old" vdpau parts. (In fact, Libav is about to
deprecate and remove the "old" API without deprecation grace period,
so we have to support it now. Moreover, there will probably be no Libav
release which supports both, so the transition is even less smooth than
we could hope, and we have to support both the old and new API.)
Whether the old or new API is used is checked by a configure test: if
the new API is found, it is used, otherwise the old API is assumed.
Some details might be handled differently. Especially display preemption
is a bit problematic with the "new" libavcodec vdpau support: it wants
to keep a pointer to a specific vdpau API function (which can be driver
specific, because preemption might switch drivers). Also, surface IDs
are now directly stored in AVFrames (and mp_images), so they can't be
forced to VDP_INVALID_HANDLE on preemption. (This changes even with
older libavcodec versions, because mp_image always uses the newer
representation to make vo_vdpau.c simpler.)
Decoder initialization in the new code tries to deal with codec
profiles, while the old code always uses the highest profile per codec.
Surface allocation changes. Since the decoder won't call config() in
vo_vdpau.c on video size change anymore, we allow allocating surfaces
of arbitrary size instead of locking it to what the VO was configured.
The non-hwdec code also has slightly different allocation behavior now.
Enabling the old vdpau special decoders via e.g. --vd=lavc:h264_vdpau
doesn't work anymore (a warning suggesting the --hwdec option is
printed instead).
* ao_coreaudio_utils: contains several utility function
* ao_coreaudio_properties: contains functions to set and get audio object
properties.
Conflicts:
audio/out/ao_coreaudio.c
Get rid of the strange and messy reliance on DEMUXER_TYPE_ constants.
Instead of having two open functions for the demuxer callbacks (which
somehow are both optional, but you can also decide to implement both...),
just have one function. This function takes a parameter that tells the
demuxer how strictly it should check for the file headers. This is a
nice simplification and allows more flexibility.
Remove the file extension code. This literally did nothing (anymore).
Change demux_lavf so that we check our other builtin demuxers first
before libavformat tries to guess by file extension.
Delete demux_avi, demux_asf, demux_mpg, demux_ts. libavformat does
better than them (except in rare corner cases), and the demuxers have
a bad influence on the rest of the code. Often they don't output
proper packets, and require additional audio and video parsing. Most
work only in --no-correct-pts mode.
Remove them to facilitate further cleanups.
Not sure how this worked. Only af_export.c and tvi_v4l2.c were
using mmap, but they didn't include osdep/mmap.h or mmap_anon.h. In
any case, we trust that the target system is sufficiently POSIX
compliant if mmap is actually defined (as checked by configure).
stream_vstream.c in particular was actually dependent on the network
code, and didn't compile anymore.
Cleanup the protocol list in mpv.rst, and add some missing ones
supported by libavformat to stream_lavf.c.
This commit removes the "old" networking code in favor of libavformat's
code.
The code was still used for mp_http, udp, ftp, cddb. http has been
mapped to libavformat's http support since approximately 6 months ago.
udp and ftp have support in ffmpeg (though ftp was added only last
month). cddb support is removed with this commit - it's probably not
important and rarely used if at all, so we don't care about it.
For some reason mp_fifo specifically handled double clicks, and other
than that was a pointless wrapper around input.c functionality.
Move the double click handling into input.c, and get rid of mp_fifo. Add
some compatibility wrappers, because so much VO code uses these
functions. Where struct mp_fifo is still used it's just a casted
struct input_ctx.
This adds support for libquvi 0.9.x, and these features:
- start time (part of youtube URL)
- youtube subtitles
- alternative source switching ('l' and 'L' keys)
- youtube playlists
Note that libquvi 0.9 is still in development. Although this seems to
be API stable now, it looks like there will be a 1.0 release, which is
supposed to be the next stable release and the actual successor of
libquvi 0.4.x.
Before this commit, SRT demuxing and display actually happened to work
on Libav. But it was using the libavcodec srt converter (which is
essentially unmaintained in Libav), and timing postprocessing didn't
work. For some background explanations see sd_lavf_srt.c.
This code was once part of subreader.c, then traveled to libass, and now
made its way back to the fork of the fork of the original code, MPlayer.
It works pretty much the same as subreader.c, except that we have to
concatenate some packets to do auto-detection. This is rather annoying,
but for all we know the actual source file could be a binary format.
Unlike subreader.c, the iconv context is reopened on each packet. This
is simpler, and with respect to multibyte encodings, more robust.
Reopening is probably not a very fast, but I suspect subtitle charset
conversion is not an operation that happens often or has to be fast.
Also, this auto-detection is disabled for microdvd - this is the only
format we know that has binary data in its packets, but is actually
decoded to text. FFmpeg doesn't really allow us to solve this properly,
because a) the input packets can be binary, and b) the output will be
checked whether it's UTF-8, and if it's not, the output is thrown away
and an error message is printed. We could just recode the decoded
subtitles before sd_ass if it weren't for that.
demux_libass.c allows us to make subtitle format detection part of the
normal file loading process. libass has no probe function, but trying to
load the start of a file (the first 4 KB) is good enough. Hope that
libass can even handle random binary input gracefully without printing
stupid log messages, and that the libass parser doesn't accept too many
non-ASS files as input.
This doesn't handle the -subcp option correctly yet. This will be fixed
later.
Currently every single AO was implementing it's own ringbuffer, many times
with slightly different semantics. This is an attempt to fix the problem.
I stole some good ideas from ao_portaudio's ringbuffer and went from there.
The main difference is this one stores wpos and rpos which are absolute
positions in an "infinite" buffer. To find the actual position for writing /
reading just apply modulo size.
The producer only modifies wpos while the consumer only modifies rpos. This
makes it pretty easy to reason about and make the operations thread safe by
using barriers (thread safety is guaranteed only in the Single-Producer/Single-
Consumer case).
Also adapted ao_coreaudio to use this ringbuffer.
This is hopefully the start of something good. ca_ringbuffer_read and
ca_ringbuffer_write can probably cleaned up from all the NULL checks once
ao_coreaudio.c gets simplyfied.
Conflicts:
audio/out/ao_coreaudio.c
This branch heavily refactors the subtitle code (both loading and
rendering), and adds support for a few new formats through FFmpeg.
We don't remove any of the old code yet. There are still some subtleties
related to subreader.c to be resolved: code page detection & conversion,
timing post-processing, UTF-16 subtitle support, support for the -subfps
option. Also, SRT reading and loading ASS via libass should be turned
into proper demuxers. (SRT is needed because Libav's is gravely broken,
and we want ASS loading via libass to cover full libass format support.
Both should be demuxers which are probed _before_ libavformat, so that
all subtitles can be loaded through the demuxer infrastructure, and
libavformat subtitles don't need to be treated in a special way.)
This allows using some formats that were not supported until now, like
WebVTT.
We still prefer the internal subtitle reader (subreader.c), because
1. Libav, and 2. random things which we probably want to keep, such as
control over formatting, codepage stuff, or various mysterious
postprecessing done in that code.
After killing the non functional AR support in c8fd9e5 I got much complaints so
this adds AR support back in (and it works). I am using the HIDRemote class by
Felix Schwarz and that part of the code is under the BSD license. I slightly
modified it replacing [NSApplication sharedApplication] with NSApp. The code
of the class is quite complex (probably because it had to deal with all the
edge cases with IOKit) but it works nicely as a black box.
In a later commit I'll remove the deprecation warnings caused by HIDRemote's
usage of Gestalt.
Check out `etc/input.conf` for the default bindings.
Apple Remote functionality is automatically compiled in when cocoa is enabled.
It can be disabled at runtime with the `--no-ar` option.
On OSX with Cocoa enabled keyDown events are now handled with
addLocalMonitorForEventsMatchingMask:handler:. This allows to respond to
events even when there is no VO initialized but the GUI is focused.
Add a basic infrastructure for subtitle converters. These converters
work sort-of like decoders, except that they produce packets instead
of subtitle bitmaps. They are put in front of actual decoders.
Start with sd_movtext. 4 lines of code are blown up to a 55 lines file,
but fortunately this is not going to be that bad for the following
converters.
Subtitle files are opened in mplayer.c, not using the demuxer
infrastructure in general. Pretend that this is not the case (outside of
the loading code) by opening a pseudo demuxer that does nothing. One
advantage is that the initialization code is now the same, and there's
no confusion about what the difference between track->stream,
track->sh_sub and mpctx->sh_sub is supposed to be.
This is a bit stupid, and it would be much better if there were proper
subtitle demuxers (there are many in recent FFmpeg, but not Libav). So
for now this is just a transition to a more proper architecture. Look
at demux_sub like an artifical limb: it's ugly, but don't hate it - it
helps you to get on with your life.
This unifies the subtitle rendering path. Now all subtitle rendering
goes through sd_ass.c/sd_lavc.c/sd_spu.c.
Before that commit, the spudec.h functions were used directly in
mplayer.c, which introduced many special cases. Add sd_spu.c, which is
just a small wrapper connecting the new subtitle render API with the
dusty old vobsub decoder in spudec.c.
One detail that changes is that we always pass the palette as extra
data, instead of passing the libdvdread palette as pointer to spudec
directly. This is a bit roundabout, but actually makes the code simpler
and more elegant: the difference between DVD and non-DVD dvdsubs is
reduced.
Ideally, we would just delete spudec.c and use libavcodec's DVD sub
decoder. However, DVD playback with demux_mpg produces packets
incompatible to lavc. There are incompatibilities the other way around
as well: packets from libavformat's vobsub demuxer are incompatible to
spudec.c. So we define a new subtitle codec name for demux_mpg subs,
"dvd_subtitle_mpg", which only sd_spu can decode.
There is actually code in spudec.c to "assemble" fragments into complete
packets, but using the whole spudec.c is easier than trying to move this
code into demux_mpg to fix subtitle packets.
As additional complication, Libav 9.x can't decode DVD subs correctly,
so use sd_spu in that case as well.
The -no-ass switch used to disable any use of libass for text subtitles.
This is not really the case anymore, because libass is now always
involved when rendering text. The only remaining use of -no-ass is
disabling styling or showing subtitles on the terminal. On the other
hand, the old subtitle rendering path is a big reason why the subtitle
code is still a big mess with an awful number of obscure special cases.
In order to simplify it, remove the old subtitle rendering code, and
always go through sd_ass.c. Basically, we use ASS_Track as central data
structure for storing text subtitles instead of struct sub_data. This
also makes libass mandatory for all text subs, even if they are printed
to the terminal in -no-video mode. (We could add something like sd_text
to avoid this, but it's not worth the trouble.)
struct sub_data and subreader.c are still around, even its ASS/SSA
reader. But struct sub_data is freed right after converting it to
ASS_Track. The internal ASS reader actually can handle some obscure
cases libass can't, like files encoded in UTF-16.
Stefano Pigozzi
wm4
James Ross-Gowan
Diogo Franco (Kovensky)
Rudolf Polzer
Jonathan Yong