This is mainly for the nnedi3 user shader. With all whose NN weights
hardcoded into the shader source code, the shader file could be as
large as 300 kB.
User hooks can now use an extra WHEN expression to specify when the
shader should be run. For example, this can be used to only run a chroma
scaling shader `WHEN CHROMA.w LUMA.w <`.
There's a slight semantics change to user shaders: When trying to bind a
texture that does not exist, a shader will now be silently skipped
(similar to when the condition is false) instead of generating an error.
This allows shader stages to depend on an optional earlier stage without
having to copy/paste the same condition everywhere.
(In other words: there's an implicit condition on all of the bound
textures existing)
The default behavior of vo_opengl has pretty much always been 'show the
source colors as-is, without caring to adapt it to the target device'.
This decision is mostly based on the fact that if we do anything else,
lots of people will complain.
With the rise of content like BT.2020, however, it turns out more people
complain about this content being very desaturated than people complain
about this content not matching VLC - so let's just map ultra-wide gamut
content back down to standard gamut by default.
Instead of measuring the actual upload time, this instead measures the
time needed to render + map the texture via vdpau. These numbers are
still useful, since they're part of the critical path.
This is plumbed through a new VOCTRL, VOCTRL_PERFORMANCE_DATA, and
exposed as properties render-time-last, render-time-avg etc.
All of these numbers are in microseconds, which gives a good precision
range when just outputting them via show-text. (Lua scripts can
obviously still do their own formatting etc.)
Signed-off-by: wm4 <wm4@nowhere>
To avoid blocking the CPU, we use 8 time objects and rotate through
them, only blocking until the last possible moment (before we need
access to them on the next iteration through the ring buffer). I tested
it out on my machine and 4 query objects were enough to guarantee
block-free querying, but the extra margin shouldn't hurt.
Frame render times are just output at the end of each frame, via MP_DBG.
This might be improved in the future. (In particular, I want to expose
these numbers as properties so that users get some more visible feedback
about render times)
Currently, we measure pass_render_frame and pass_draw_to_screen
separately because the former might be called multiple times due to
interpolation. Doing it this way gives more faithful numbers. Same goes
for frame upload times.
Enable m_sub_options_copy() to copy nested sub-options, and also enable
it to create an option struct from defaults. We can get rid of most of
the crap in assign_options() now.
Calling handle_scaler_opt() to get a static allocation for scaler name
is still needed. It's moved to reinit_scaler(), which seems to be a
better place for it. Without it, dangling pointers could be created when
options are changed. (And in fact, this fixes possible dangling pointers
for window.name.) In theory we could create a dynamic copy, but that
seemed even more messy.
Chance of regressions.
Commit 026b75e7 actually enabled changing icc options at runtime (via
vo_cmdline), but it didn't quite work. In particular, changing the icc-
profile option just kept the old profile, because it was cached
accordingly.
As part of this, change gl_lcms.opts from a struct to a pointer to a
struct. We properly copy it, instead of allowing possibly dangling
strings, like it was done in a working but unclean way before.
Also, reinit the whole rendering chain when the auto icc profile
changes, just like it's done when icc options are changed.
Passing the bstr thing as pointer makes no sense. Everywhere else bstr
structs are passed by value because they're so small. Only when it's
supposed to receive a return value they're not.
Originally, video.c did not access any CMS things (other than lut3d
being set on it), but this has changed. In practice, almost all accesses
to it have moved to video.c. vo_opengl only created it, and set the auto
icc profile path.
Complete the move.
Some things wrt. option handling are a bit fishy. (But when is this not
the case.)
icc-profile-auto was not tested, but the distributed human CI will take
care of it.
This algorithm works really well. Setting it is a much better
"out-of-the-box" experience than just clipping, which will always look
ugly.
In other words, with this default, users of mpv will just be able to
play HDR content without even realizing it's HDR (pretty much).
Instead of doing HDR tone mapping on an ad-hoc basis inside
pass_colormanage, the reference peak of an image is now part of the
image params (alongside colorspace, gamma, etc.) and tone mapping is
done whenever peak_src != peak_dst.
To get sensible behavior when mixing HDR and SDR content and displays,
target-brightness is a generic filler for "the assumed brightness of SDR
content".
This gets rid of the weird display_scaled hack, sets the framework
for multiple HDR functions with difference reference peaks, and allows
us to (in a future commit) autodetect the right source peak from
the HDR metadata.
(Apart from metadata, the source peak can also be controlled via
vf_format. For HDR content this adjusts the overall image brightness,
for SDR content it's like simulating a different exposure)
This requires the GL_EXT_texture_norm16 extension and works in ANGLE.
A default precision had to be set for sampler3Ds, otherwise the shaders
would fail to compile.
Rename it to get out of OpenGL's namespace. The gl_ prefix is used by
other mpv functions, but no OpenGL ones.
The "slice" parameter was never actually used, and all callers passed 0
for it.
For some reason, GLES has no glMapBuffer, only glMapBufferRange.
GLES 2 has no buffer mapping at all, and GL 2.1 does not always have
glMapBufferRange. On those PBOs remain unsupported (there's no reason to
care about GL 2.1 without the extension).
This doesn't actually work on ANGLE, and I have no idea why. (There are
artifacts on OSD, as if parts of the OSD data weren't copied.) It works
on desktop OpenGL and at least 1 other ES 3 implementation. Don't enable
it on ANGLE, I guess.
Following commit 84ccebd9, the internal helpers don't allow GL_RGB and
GL_RGBA as internal formats for FBO attachments anymore.
While OpenGL itself is perfectly fine with it, I don't see much of a
reason to bother, and mixing sized and unsized internal formats is
confusing anyway.
Just remove these formats.
gl_video_upload_image() can fail in the hardware decoding case. In this
case rendering continued "normally", which meant that pass_get_img_tex()
would kill the process with an assertion failure.
Fix this by allowing gl_video_upload_image() to fail, and exit rendering
early enough to skip code which requires an image to be present. (Maybe
this is still a bit too subtle, but better than before.)
Set an error flag, and render the blue screen we introduced for shader
errors. (For this purpose also move the rendering of it to final output,
to ensure it's visible at all.) The error flag is temporary, because the
associated failure might also be temporary, unlike shader compilation
errors.
ANGLE doesn't handle this very strictly. But if they change this in the
future, it shouldn't brick us.
Not quite happy with this glsl_extensions fields, but it is quite
unintrusive after all.
With the new hooks mechanism, user shaders and such are actually loaded
before rendering starts, instead of being loaded during rendering. This
is used to cache them (instead of e.g. reparsing them every frame).
The cached state wasn't cleared correctly in some situations. Namely,
resizing didn't correctly enable/disable prescale hooks.
Reorganize how these reinitializations are handled. Get rid of
reinit_rendering(), whose meaning was pretty unclear. Call the required
functions to reset or recreate state directly wherever they are needed.
This makes it so that users with actual HDR displays can just set their
config to target-trc=st2084 and get native HDR output. This will look a
bit silly for SDR content (everything will be really bright), but for
lack of a better tone mapping situation (including reverse tone mapping)
this is the easiest thing to do for now.
Ideally the brightness metadata should be part of the colorspace struct
or something (with mpv always adapting where necessary), but it depends
on the TRC and not the primaries so it's a bit more complicated than
that.
Since dumb mode is affected by tone mapping (which I'll call a feature,
not a bug), we need to copy over the configuration - in particular, the
defaults. (To prevent a render failure)
Since HDR content is now auto-detected as such, we should probably do
something smarter in the "no configuration" case, such as outputting
gamma 2.2 instead.
This decision will affect the majority of users of stock configurations
who just play back appropriately tagged HDR files, so having a good
default behavior is important. "Output the HDR content as-is" is
definitely not likely to give the user a good result.
Make it dynamic and never remove entries from it.
For now, this is better than possibly creating dangling pointers all
over the place in the gl_user_shader struct.
Untested.
This is now a configurable option, with tunable parameters.
I got inspiration for these algorithms off wikipedia. "simple" seems to
work pretty well, but not well enough to make it a reasonable default.
Some other notable candidates:
- Local functions (e.g. based on local contrast or gradient)
- Clamp with soft knee (linear up to a point)
- Mapping in CIE L*Ch. Map L smoothly, clamp C and h.
- Color appearance models
These will have to be implemented some other time.
Note that the parameter "peak_src" to pass_tone_map should, in
principle, be auto-detected from the SEI information of the source file
where available. This will also have to be implemented in a later
commit.
Due to the way color management in mpv worked historically, the subtitle
blending function was written to preserve the linearity of the input.
(In the past, the 3DLUT function required linear inputs)
Since the 3DLUT was refactored to accept the video color directly, the
re-linearization after blending is now virtually always redundant.
(Notably, it's also redundant when CMS is turned off, so this way of
writing the code stopped making sense a long time ago. It is a remnant
from before the pass_colormanage function was as flexible as it is now)
Currently, this relies on the user manually entering their display
brightness (since we have no way to detect this at runtime or from ICC
metadata). The default value of 250 was picked by looking at ~10 reviews
on tftcentral.co.uk and realizing they all come with around 250 cd/m^2
out of the box. (In addition, ITU-R Rec. BT.2022 supports this)
Since there is no metadata in FFmpeg to indicate usage of this TRC, the
only way to actually play HDR content currently is to set
``--vf=format=gamma=st2084``. (It could be guessed based on SEI, but
this is not implemented yet)
Incidentally, since SEI is ignored, it's currently assumed that all
content is scaled to 10,000 cd/m^2 (and hard-clipped where out of
range). I don't see this assumption changing much, though.
As an unfortunate consequence of the fact that we don't know the display
brightness, mixed with the fact that LittleCMS' parametric tone curves
are not flexible enough to support PQ, we have to build the 3DLUT
against gamma 2.2 if it's used. This might be a good thing, though,
consdering the PQ source space is probably not fantastic for
interpolation either way.
Partially addresses #2572.
This is much more readable than hard-coding magic IDs all over the file,
and removes the need for all the explanatory comments that were a direct
result of this.
This macro takes care of rotation, swizzling, integer conversion and
normalization automatically. I found the performance impact to be
nonexistant for superxbr and debanding, although rotation *did* have an
impact due to the extra matrix multiplication. (So it gets skipped where
possible)
All of the internal hooks have been rewritten to use this new mechanism,
and the prescaler hooks have finally been separated from each other.
This also means the prescale FBO kludge is no longer required.
This fixes image corruption for image formats like 0bgr, and also fixes
prescaling under rotation. (As well as other user hooks that have
orientation-dependent access)
The "raw" attributes (tex, tex_pos, pixel_size) are still un-rotated, in
case something needs them, but ideally the hooks should be rewritten to
use the new API as much as possible. The hooked texture has been renamed
from just NAME to NAME_raw to make script authors notice the change (and
also deemphasize direct texture access).
This is also a step towards getting rid of the use_integer pass.
This replaces the previous TRANSFORM by WIDTH, HEIGHT and OFFSET where
WIDTH and HEIGHT are RPN expressions. This allows for more fine-grained
control over the output size, and also makes sure that overwriting
existing textures works more cleanly.
(Also add some more useful bstr functions)
This allows users to add their own near-arbitrary hooks to the vo_opengl
processing pipeline, greatly enhancing the flexibility of user shaders.
This enables, among other things, user shaders such as CrossBilateral,
SuperRes, LumaSharpen and many more.
To make parsing the user shaders easier, shaders are now loaded as
bstrs, and the hooks are set up during video reconfig instead of on
every single frame.
These are "sequence points" where the image could be rendered out to an
FBO, hooked, and re-loaded if any such hook exists. This is perfect for
things like the current user shaders system, as well as optional effects
like unsharp masking.
Note that since we have to pick *some* FBO to store the optionally
hooked texture, we just store it in an array indexed by an increasing
counter. Since we only ever store as many as MAX_TEXTURE_HOOKS + all
internal hook points entries, this is guaranteed to be enough space.
This commit also removes some of the now unused FBOs.
The hook mechanism allows arbitrary processing stages to get dispatched
whenever certain named textures have been "finalized" by the code.
This is mostly meant to serve as a change that opens up the internal
processing in pass_read_video to user scripts, but as a side benefit all
of the code dealing with offsets and plane alignment and other such
confusing things has been rewritten.
This hook mechanism is powerful enough to cover the needs of both
debanding and prescaling (and more), so as a result they can be removed
from pass_read_video entirely and implemented through hooks.
Some avenues for optimization:
- The prescale hook is currently somewhat distributed code-wise. It might be
cleaner to split it into superxbr and NNEDI3 hooks which can each be
self-contained.
- It might be possible to move a large part of the hook code out to an
external file (including the hook definitions for debanding and
prescaling), which would be very much desired.
- Currently, some stages (chroma merging, integer conversion) will
*always* run even if unnecessary. I'm planning another series of
refactors (deferred img_tex) to allow dropping unnecessary shader
stages like these, but that's probably some ways away. In the meantime
it would be doable to re-add some of the logic to skip these stages if
we know we don't need them.
- More hook locations could be added (?)
Instead of rounding down, we round to the nearest float. This reduces
the maximum possible error introduced by this rounding operation. Also
clarify the comment.
Fixes broken colors with --vf=format=0bgr (but only if deband is
disabled).
0bgr means the first byte is padding, while the following three bytes
are bgr. From the vo_opengl perspective, it has 4 physical components
with 3 logical components. copy_img_tex() simply copied 3 components
from the physical representation, which means the last component (r) was
sliced off.
Fix this by not using p->color_swizzle for packed formats, and instead
let packed formats set the per-plane swizzle in texplane.swizzle. The
latter applies the swizzle as part of operation in copy_img_tex(), which
essentially moves physical to logical representations.
Unfortunately, debanding (and thus with opengl-hq defaults) is still
broken.
Make the find_plane_format function take a bit count.
This also makes the function's comment true for the first time the
function and its comment exist. (It was commented as taking bits, but
always took bytes.)
Now that we know in advance whether an implementation should support a
specific format, we have more flexibility when determining which format
to use.
In particular, we can drop the roundabout ES logic.
I'm not sure if actually trying to create the FBO for probing still has
any value. But it might, so leave it for now.
Even if everything else is available, the need for first class arrays
breaks it. In theory we could fix this since we don't strictly need
them, but I guess it's not worth bothering.
Also give the misnamed have_mix variable a slightly better name.
This merges all knowledge about texture format into a central table.
Most of the work done here is actually identifying which formats exactly
are supported by OpenGL(ES) under which circumstances, and keeping this
information in the format table in a somewhat declarative way. (Although
only to the extend needed by mpv.) In particular, ES and float formats
are a horrible mess.
Again this is a big refactor that might cause regression on "obscure"
configurations.
