The intention was that if --blend-subtitles is enabled, the frame should
always be re-rendered instead of using e.g. a cached scaled frame. The
reason is that subtitles can change anyway, e.g. if you pause and change
subtitle size and such.
On the other hand, if the frame is marked as repeated, it should always
use the cached copy. Actually "simplify" this and drop the cache only if
playback is paused (which frame->still indicates indirectly).
Also see PR #3773.
unmap_current_image() is called after rendering. This essentially
invalidates the textures, so we can't assume that the image is still
present.
Also see PR #3773.
This allows us to define the tukey window (and other tapered windows).
Also add a missing option definition for `wblur` while we're at it, to
make testing out window-related stuff easier.
It's not that easy to decide whether a frame needs to be
reuploaded/rerendered. Using unique frame IDs for input makes it
slightly easier and more robust. This also removes the use of video PTS
in the interpolation path.
This should also avoid reuploading the video frame if it's just redrawn
in paused mode, or when using OSD/subtitles in cover art mode.
It turns out the glFlush() call really helps in some cases, though only
in audio timing mode (where we render, then wait for a while, then
display the frame). Add a --opengl-early-flush=auto mode, which does
exactly that.
It's unclear whether this is fine on OSX (strange things going on
there), but it should be.
See #3670.
In "dumb mode" (where most features are disabled and which only performs
some basic rendering) we explicitly copy a set of whitelisted options,
and leave all the other options at their default values. Add the new
--opengl-early-flush option to this whitelist. Also remove an option
field accidentally added in the commit adding --opengl-early-flush.
It seems this can cause issues with certain platforms, so better to
disable it by default. The original reason for this isn't overly
justified, and display-sync mode should get rid of the need for it
anyway.
The new option is meant for testing, and will probably be removed if
nobody comes up and reports that enabling the option actually improves
anything.
Other than being overly convoluted, this seems to make sense to me.
Except that to get the "rot" transform I have to set flip=true, which
makes no sense at all to me.
Combining rotation and cropping didn't work. It was just completely
broken.
I'm still not sure if this is correct. Chroma positioning seems to be
broken on rotation. There might also be a problem with non-mod-2 frame
sizes. Still, strictly an improvement for both rotated and non-rotated
rendering modes.
Also, this could probably be written in a more elegant way.
When we rotate the inmage by 90° or 270°, chroma width and height need
to be swapped.
Fixes#3568.
But is the chroma sub location correct? Who the hell knows...
Extend the flag-based notification mechanism that was used via
M_OPT_TERM. Make the vo_opengl update mechanism use this (which, btw.,
also fixes compilation with OpenGL renderers forcibly disabled).
While this adds a 3rd mechanism and just seems to further the chaos, I'd
rather have a very simple mechanism now, than actually furthering the
mess by mixing old and new update mechanisms. In particular, we'll be
able to remove quite some property implementations, and replace them
with much simpler update handling. The new update mechanism can also
more easily refactored once we have a final mechanism that handles
everything in an uniform way.
There were multiple values under M_OPT_EXIT (M_OPT_EXIT-n for n>=0).
Somehow M_OPT_EXIT-n either meant error code n (with n==0 no error?), or
the number of option valus consumed (0 or 1). The latter is MPlayer
legacy, which left it to the option type parsers to determine whether an
option took a value or not. All of this was changed in mpv, by requiring
the user to use explicit syntax ("--opt=val" instead of "-opt val").
In any case, the n value wasn't even used (anymore), so rip this all
out. Now M_OPT_EXIT-1 doesn't mean anything, and could be used by a new
error code.
Negative height is used to signal a flipped framebuffer. There's
absolutely no reason to pass this down to overlay_adjust(), and only
requires implementers to deal with an additional special-case.
A minor cleanup that makes the code simpler, and guarantees that we
cleanup the GL state properly at any point.
We do this by reusing the uniform caching, and assigning each sampler
uniform its own texture unit by incrementing a counter. This has various
subtle consequences for the GL driver, which hopefully don't matter. For
example, it will bind fewer textures at a time, but also rebind them
more often.
For some reason we keep TEXUNIT_VIDEO_NUM, because it limits the number
of hook passes that can be bound at the same time.
OSD rendering is an exception: we do many passes with the same shader,
and rebinding the texture each pass. For now, this is handled in an
unclean way, and we make the shader cache reserve texture unit 0 for the
OSD texture. At a later point, we should allocate that one dynamically
too, and just pass the texture unit to the OSD rendering code. Right now
I feel like vo_rpi.c (may it rot in hell) is in the way.
The caller now has to call gl_sc_reset(), and _after_ rendering. This
way we can unset OpenGL state that was setup for rendering. This affects
the shader program, for example. The next commit uses this to
automatically manage texture units via the shader cache.
vo_rpi.c changes untested.
The wrong enum got copied here, so it was essentially using the transfer
characteristics as the primaries (instead of the primaries), which
accidentally worked fine most of the time (since the two usually
coincided), but broke on weird/mistagged files.
This overlay support specifically skips the OpenGL rendering chain, and
uses GL rendering only for OSD/subtitles. This is for devices which
don't have performant GL support.
hwdec_rpi.c contains code ported from vo_rpi.c. vo_rpi.c is going to be
deprecated. I left in the code for uploading sw surfaces (as it might
be slightly more efficient for rendering sw decoded video), although
it's dead code for now.
Just another corner-caseish potential issue. Unlike unreffing the image
manually, unref_current_image() also takes care of properly unmapping
hwdec frames. (The corner-case part of this is that it's probably never
mapped at this point, but it's apparently not entirely guaranteed.)
The " || vimg->mpi" part virtually never seems to trigger, but on the
other hand could possibly create unintended corner cases (for example by
trying to upload a NULL image, which would then be marked as an error
and render a blue screen).
I guess it's a leftover from over times, where a NULL image meant
"redraw the current frame". This is now handled by actually passing
along the current frame.
Instead of copying the options around... just don't. video.c now has
full control over when options are updated. (It still gets notified from
outside, but it decides when the updated options are copied: when
m_config_cache_update() is called.) So there's no need for tricky
stuff, and it can be simplified a bit.
Also change lcms.c. We could do it like video.c, and get the options
from the global config store. But it seems simpler to just provide a
pointer to an option struct, which is arbitrarily mutated from the
outside (from the perspective of lcms.c).
Setting --icc-profile had no effect, until a vo_opengl option was
changed at runtime. We must initialize the renderer for the initial
option state too.
For some reason, the ICC profile gets loaded twice. The next commit
happens to fix this.
vo_opengl sub-option were always rather annoying to handle. It seems
better to make them global options instead. This is simpler and easier
to use. The only disadvantage we are aware of is that it's not clear
that many/all of these new global options work with vo_opengl only.
--vo=opengl-hq is also deprecated.
There is extensive compatibility with the old behavior. One exception is
that --vo-defaults will not apply to opengl-hq (though with opengl it
still works). vo-cmdline is also dysfunctional and will be removed in a
following commit.
These changes also affect opengl-cb.
The update mechanism is still rather inefficient: it requires syncing
with the VO after each option change, rather than batching updates.
There's also no granularity (video.c just updates "everything", and if
auto-ICC profiles are enabled, vo_opengl.c will fetch them on each
update).
Most of the manpage changes were done by Niklas Haas <git@haasn.xyz>.
Reduce accesses to the renderer opts in vo_opengl.c, and instead add
accessors for them to video.c.
I suppose gamma and maybe icc-auto could be moved to vo_opengl.c
options. Also, the output colorspace could probably be adjusted to what
is really used, not just the options (although it's possible that this
commit changes this, due to video.c mutating its own copy of the options
according to actual renderer capapbilities).
But don't deal with this now.
Deprecated in favor of user-shaders, which are functionally equivalent
but superior. (Except in the case of scaler-shader, which has no direct
replacement, but it turned out to be a very unpopular feature either way
- most custom scalers don't fit into the mpv kernel infrastructure and
are therefore implemented as user shaders either way)
Signed-off-by: wm4 <wm4@nowhere>
It seems like many GL implementations (including Mesa) choke on this,
while others are fine. We still think that this use of the GL API is
allowed by the standard (at least in the Mesa case), so to reduce
confusion, explicitly check the "controversial" calls, and use an
appropriate error message.
This requires changing the pixel upload alignment because the odd sizes
might not be aligned to multiples of 4.
Anyway, the restriction has no real benefit and the sizes in between 32
and 64 might be worth using, so just drop it.
This code had the exact same texture indexing bug that the original
scaler code had before the introduction of the LUT_POS macro to fix it.
We can re-use this same macro here, and the performance drop is
virtually entirely negligible. The benefit is greatly improved LUT
accuracy as the 3DLUT size decreases - in particular, the old LUT
started introducing more and more black crush the lower your LUT size is
(because the error was essentially an over-contrast bias, with a
magnitude linearly related to the lut size).
The new code improves black stability as the LUT size decreases, and
only at very low values (16 and below) do black levels start noticeably
getting affected (due to crude linearization of the nonlinear response
curve).
The default value of 3dlut-size is definitely generous enough for this
to make no difference out of the box, but it also causes no performance
drop at all on my machine so I see no harm in improving the logic.
Furthermore, this means we could easily decrease the default 3dlut size
in a future commit, perhaps even down to 64x64x64 as a default. (But
more testing is warranted here)
This uses GLSL mix() instead of going through an indirect texture
access. Easy to implement and might require less resources on some
devices, since the oversample code was already essentially just a
special case of this.
Could be made the new default (as per issue #2685), but that should be
done in a separate commit.
This can for example happen with vo_opengl_cb, if it is used with a GL
implementation that does not supports FBOs. (mpv itself should never
attempt to use FBOs if they're not available.)
Without this check it would trigger an assert() in our dummy
glBindFramebuffer wrapper.
Suspected cause of #3308, although it's still unlikely.
This moves some of the bulky user-shader specific logic into the file
dedicated to it. Rather than expose video.c state, variable lookup is
now done via a simulated closure.
This involves multiple changes:
1. Brightness metadata is split into nominal peak and signal peak.
For a quick and dirty explanation: nominal peak is the brightest value
that your color space can represent (i.e. the brightness of an encoded
1.0), and signal peak is the brightest value that actually occurs in
the video (i.e. the brightest thing that's displayed).
2. vo_opengl uses a new decision logic to figure out the right nom_peak
and sig_peak for all situations. It also does a better job of picking
the right target gamut/colorspace to use for the OSD. (Which still is
and still should be treated as sRGB). This change in logic also
fixes#3293 en passant.
3. Since it was growing rapidly, the logic for auto-guessing / inferring
the right colorimetry configuration (in pass_colormanage) was split from
the logic for actually performing the adaptation (now pass_color_map).
Right now, the new logic doesn't do a whole lot since HDR metadata is
still ignored (but not for long).
This has two reasons:
1. I tend to add new fields to this metadata, and every time I've done
so I've consistently forgotten to update all of the dozens of places in
which this colorimetry metadata might end up getting used. While most
usages don't really care about most of the metadata, sometimes the
intend was simply to “copy” the colorimetry metadata from one struct to
another. With this being inside a substruct, those lines of code can now
simply read a.color = b.color without having to care about added or
removed fields.
2. It makes the type definitions nicer for upcoming refactors.
In going through all of the usages, I also expanded a few where I felt
that omitting the “young” fields was a bug.
Commit 883d3114 seems to have (accidentally?) dropped the FBOTEX_FUZZY
from the output_fbo resize, which means that current master will keep
resizing and resizing the FBO as you change the window size, introducing
severe memory leaking after a while. (Not sure why that would cause
memory leaks, but I blame nvidia)
Either way, it's bad for performance too, so it's worth fixing.
GL generally does not support flipping the image on upload, meaning
negative strides are not supported. vo_opengl handles this by flipping
rendering if the stride is inverted, and gl_pbo_upload() "ignores"
negative strides by uploading without flipping the image.
If individual planes had strides with different signs, this broke. The
flipping affected the entire image, and only the sign of the first plane
was respected.
This is just a crazy corner case that will never happen, but it turns
out this is quite simple to support, and actually improves the code
somewhat.
This introduces a gl_pbo_upload_tex() function, which works almost like
our gl_upload_tex() glTexSubImage2D() wrapper, except it takes a struct
which caches the PBO handles. It also takes the full texture size (to
make allocating an ideal buffer size easier), and a parameter to disable
PBOs (so that the caller doesn't have to duplicate the gl_upload_tex()
call if PBOs are disabled or unavailable).
This also removes warnings and fallbacks on PBO failure. We just
silently try using PBOs on every frame, and if that fails at some point,
revert to normal texture uploads. Probably doesn't matter.
Instead of hard-coding a big list, move some of the functionality
to csputils. Affects both the auto-guess blacklist and the peak
estimation.
Also update the comments.
Too many "exceptions" these days, it's easier to just hard-code a
whitelist instead of a blacklist. And besides, it only really makes
sense to avoid adaptation for BT.601 specifically, since that's the one
we auto-guess based on the resolution.
I'm not even sure why we ever consulted *_src to begin with, since that
just describes the current image format - and not the original metadata.
(And in fact, we specifically had logic to work around the impliciations
this had on linear scaling)
image_params is *the* authoritative source on the intended (i.e.
reference) image metadata, whereas *_src may be changed by previous
passes already. So only consult image_params for picking auto-generated
values.
Also, add some more missing "wide gamut" and "non-gamma" curves to the
autoconfig blacklist. (Maybe it would make sense to move this list to
csputils in the future? Or perhaps even auto-detect it based on the
associated primaries)
User request and not that hard. Closes#3157.
Note that FFmpeg doesn't support this and there's no signalling in HEVC
etc., so the only way users can access it is by using vf_format
manually.
Mind: This encoding uses full range values, not TV range.
This HDR function is unique in that it's still display-referred, it just
allows for values above the reference peak (super-highlights). The
official standard doesn't actually document this very well, but the
nominal peak turns out to be exactly 12.0 - so we normalize to this
value internally in mpv. (This lets us preserve the property that the
textures are encoded in the range [0,1], preventing clipping and making
the best use of an integer texture's range)
This was grouped together with SMPTE ST2084 when checking libavutil
compatibility since they were added in the same release window, in a
similar timeframe.
Until now, we've always converted vdpau video surfaces to RGB, and then
mapped the resulting RGB texture. Change this so that the surface is
mapped as NV12 plane textures.
The reason this wasn't done until now is because vdpau surfaces are
mapped in an "interlaced" way as separate fields, even for progressive
video. This requires messy reinterleraving. It turns out that even
though it's an extra processing step, the result can be faster than
going through the video mixer for RGB conversion.
Other than some potential speed-gain, doing this has multiple other
advantages. We can apply our own color conversion, which is important in
more complex cases. We can correctly apply debanding and potentially
other processing that requires chroma-specific or in-YUV handling.
If deinterlacing is enabled, this switches back to the old RGB
conversion method. Until we have at least a primitive deinterlacer in
vo_opengl, this will stay this way. The d3d11 and vaapi code paths are
similar. (Of course these don't require any crazy field reinterleaving.)
The OpenGL 3.0+ and ES specs are quite clear on what values are
accepted for the attachment object name parameter. And there's no
overlap for the default framebuffer. Sigh.
Probably fixes Mesa raising an error in this case and might fix#3251.
Regression by the previous vo_opengl change.
Until now, we've used system-specific API (GLX, EGL, etc.) to retrieve
the depth of the default framebuffer. (We equal this to display depth
and use the determined depth for dithering.)
We can actually retrieve this value through standard GL API, and it
works everywhere (except GLES 2 of course). This simplifies everything a
great deal.
egl_helpers.c is empty now. But I expect that some EGL boilerplate will
be moved to it, so don't remove it yet.
This is mainly for the nnedi3 user shader. With all whose NN weights
hardcoded into the shader source code, the shader file could be as
large as 300 kB.
User hooks can now use an extra WHEN expression to specify when the
shader should be run. For example, this can be used to only run a chroma
scaling shader `WHEN CHROMA.w LUMA.w <`.
There's a slight semantics change to user shaders: When trying to bind a
texture that does not exist, a shader will now be silently skipped
(similar to when the condition is false) instead of generating an error.
This allows shader stages to depend on an optional earlier stage without
having to copy/paste the same condition everywhere.
(In other words: there's an implicit condition on all of the bound
textures existing)
The default behavior of vo_opengl has pretty much always been 'show the
source colors as-is, without caring to adapt it to the target device'.
This decision is mostly based on the fact that if we do anything else,
lots of people will complain.
With the rise of content like BT.2020, however, it turns out more people
complain about this content being very desaturated than people complain
about this content not matching VLC - so let's just map ultra-wide gamut
content back down to standard gamut by default.
Instead of measuring the actual upload time, this instead measures the
time needed to render + map the texture via vdpau. These numbers are
still useful, since they're part of the critical path.
This is plumbed through a new VOCTRL, VOCTRL_PERFORMANCE_DATA, and
exposed as properties render-time-last, render-time-avg etc.
All of these numbers are in microseconds, which gives a good precision
range when just outputting them via show-text. (Lua scripts can
obviously still do their own formatting etc.)
Signed-off-by: wm4 <wm4@nowhere>
To avoid blocking the CPU, we use 8 time objects and rotate through
them, only blocking until the last possible moment (before we need
access to them on the next iteration through the ring buffer). I tested
it out on my machine and 4 query objects were enough to guarantee
block-free querying, but the extra margin shouldn't hurt.
Frame render times are just output at the end of each frame, via MP_DBG.
This might be improved in the future. (In particular, I want to expose
these numbers as properties so that users get some more visible feedback
about render times)
Currently, we measure pass_render_frame and pass_draw_to_screen
separately because the former might be called multiple times due to
interpolation. Doing it this way gives more faithful numbers. Same goes
for frame upload times.
Enable m_sub_options_copy() to copy nested sub-options, and also enable
it to create an option struct from defaults. We can get rid of most of
the crap in assign_options() now.
Calling handle_scaler_opt() to get a static allocation for scaler name
is still needed. It's moved to reinit_scaler(), which seems to be a
better place for it. Without it, dangling pointers could be created when
options are changed. (And in fact, this fixes possible dangling pointers
for window.name.) In theory we could create a dynamic copy, but that
seemed even more messy.
Chance of regressions.
Commit 026b75e7 actually enabled changing icc options at runtime (via
vo_cmdline), but it didn't quite work. In particular, changing the icc-
profile option just kept the old profile, because it was cached
accordingly.
As part of this, change gl_lcms.opts from a struct to a pointer to a
struct. We properly copy it, instead of allowing possibly dangling
strings, like it was done in a working but unclean way before.
Also, reinit the whole rendering chain when the auto icc profile
changes, just like it's done when icc options are changed.
Passing the bstr thing as pointer makes no sense. Everywhere else bstr
structs are passed by value because they're so small. Only when it's
supposed to receive a return value they're not.
Originally, video.c did not access any CMS things (other than lut3d
being set on it), but this has changed. In practice, almost all accesses
to it have moved to video.c. vo_opengl only created it, and set the auto
icc profile path.
Complete the move.
Some things wrt. option handling are a bit fishy. (But when is this not
the case.)
icc-profile-auto was not tested, but the distributed human CI will take
care of it.
This algorithm works really well. Setting it is a much better
"out-of-the-box" experience than just clipping, which will always look
ugly.
In other words, with this default, users of mpv will just be able to
play HDR content without even realizing it's HDR (pretty much).
Instead of doing HDR tone mapping on an ad-hoc basis inside
pass_colormanage, the reference peak of an image is now part of the
image params (alongside colorspace, gamma, etc.) and tone mapping is
done whenever peak_src != peak_dst.
To get sensible behavior when mixing HDR and SDR content and displays,
target-brightness is a generic filler for "the assumed brightness of SDR
content".
This gets rid of the weird display_scaled hack, sets the framework
for multiple HDR functions with difference reference peaks, and allows
us to (in a future commit) autodetect the right source peak from
the HDR metadata.
(Apart from metadata, the source peak can also be controlled via
vf_format. For HDR content this adjusts the overall image brightness,
for SDR content it's like simulating a different exposure)
This requires the GL_EXT_texture_norm16 extension and works in ANGLE.
A default precision had to be set for sampler3Ds, otherwise the shaders
would fail to compile.
Rename it to get out of OpenGL's namespace. The gl_ prefix is used by
other mpv functions, but no OpenGL ones.
The "slice" parameter was never actually used, and all callers passed 0
for it.
For some reason, GLES has no glMapBuffer, only glMapBufferRange.
GLES 2 has no buffer mapping at all, and GL 2.1 does not always have
glMapBufferRange. On those PBOs remain unsupported (there's no reason to
care about GL 2.1 without the extension).
This doesn't actually work on ANGLE, and I have no idea why. (There are
artifacts on OSD, as if parts of the OSD data weren't copied.) It works
on desktop OpenGL and at least 1 other ES 3 implementation. Don't enable
it on ANGLE, I guess.
Following commit 84ccebd9, the internal helpers don't allow GL_RGB and
GL_RGBA as internal formats for FBO attachments anymore.
While OpenGL itself is perfectly fine with it, I don't see much of a
reason to bother, and mixing sized and unsized internal formats is
confusing anyway.
Just remove these formats.
gl_video_upload_image() can fail in the hardware decoding case. In this
case rendering continued "normally", which meant that pass_get_img_tex()
would kill the process with an assertion failure.
Fix this by allowing gl_video_upload_image() to fail, and exit rendering
early enough to skip code which requires an image to be present. (Maybe
this is still a bit too subtle, but better than before.)
Set an error flag, and render the blue screen we introduced for shader
errors. (For this purpose also move the rendering of it to final output,
to ensure it's visible at all.) The error flag is temporary, because the
associated failure might also be temporary, unlike shader compilation
errors.
ANGLE doesn't handle this very strictly. But if they change this in the
future, it shouldn't brick us.
Not quite happy with this glsl_extensions fields, but it is quite
unintrusive after all.
With the new hooks mechanism, user shaders and such are actually loaded
before rendering starts, instead of being loaded during rendering. This
is used to cache them (instead of e.g. reparsing them every frame).
The cached state wasn't cleared correctly in some situations. Namely,
resizing didn't correctly enable/disable prescale hooks.
Reorganize how these reinitializations are handled. Get rid of
reinit_rendering(), whose meaning was pretty unclear. Call the required
functions to reset or recreate state directly wherever they are needed.
This makes it so that users with actual HDR displays can just set their
config to target-trc=st2084 and get native HDR output. This will look a
bit silly for SDR content (everything will be really bright), but for
lack of a better tone mapping situation (including reverse tone mapping)
this is the easiest thing to do for now.
Ideally the brightness metadata should be part of the colorspace struct
or something (with mpv always adapting where necessary), but it depends
on the TRC and not the primaries so it's a bit more complicated than
that.
Since dumb mode is affected by tone mapping (which I'll call a feature,
not a bug), we need to copy over the configuration - in particular, the
defaults. (To prevent a render failure)
Since HDR content is now auto-detected as such, we should probably do
something smarter in the "no configuration" case, such as outputting
gamma 2.2 instead.
This decision will affect the majority of users of stock configurations
who just play back appropriately tagged HDR files, so having a good
default behavior is important. "Output the HDR content as-is" is
definitely not likely to give the user a good result.
Make it dynamic and never remove entries from it.
For now, this is better than possibly creating dangling pointers all
over the place in the gl_user_shader struct.
Untested.
This is now a configurable option, with tunable parameters.
I got inspiration for these algorithms off wikipedia. "simple" seems to
work pretty well, but not well enough to make it a reasonable default.
Some other notable candidates:
- Local functions (e.g. based on local contrast or gradient)
- Clamp with soft knee (linear up to a point)
- Mapping in CIE L*Ch. Map L smoothly, clamp C and h.
- Color appearance models
These will have to be implemented some other time.
Note that the parameter "peak_src" to pass_tone_map should, in
principle, be auto-detected from the SEI information of the source file
where available. This will also have to be implemented in a later
commit.
Due to the way color management in mpv worked historically, the subtitle
blending function was written to preserve the linearity of the input.
(In the past, the 3DLUT function required linear inputs)
Since the 3DLUT was refactored to accept the video color directly, the
re-linearization after blending is now virtually always redundant.
(Notably, it's also redundant when CMS is turned off, so this way of
writing the code stopped making sense a long time ago. It is a remnant
from before the pass_colormanage function was as flexible as it is now)
Currently, this relies on the user manually entering their display
brightness (since we have no way to detect this at runtime or from ICC
metadata). The default value of 250 was picked by looking at ~10 reviews
on tftcentral.co.uk and realizing they all come with around 250 cd/m^2
out of the box. (In addition, ITU-R Rec. BT.2022 supports this)
Since there is no metadata in FFmpeg to indicate usage of this TRC, the
only way to actually play HDR content currently is to set
``--vf=format=gamma=st2084``. (It could be guessed based on SEI, but
this is not implemented yet)
Incidentally, since SEI is ignored, it's currently assumed that all
content is scaled to 10,000 cd/m^2 (and hard-clipped where out of
range). I don't see this assumption changing much, though.
As an unfortunate consequence of the fact that we don't know the display
brightness, mixed with the fact that LittleCMS' parametric tone curves
are not flexible enough to support PQ, we have to build the 3DLUT
against gamma 2.2 if it's used. This might be a good thing, though,
consdering the PQ source space is probably not fantastic for
interpolation either way.
Partially addresses #2572.
This is much more readable than hard-coding magic IDs all over the file,
and removes the need for all the explanatory comments that were a direct
result of this.
This macro takes care of rotation, swizzling, integer conversion and
normalization automatically. I found the performance impact to be
nonexistant for superxbr and debanding, although rotation *did* have an
impact due to the extra matrix multiplication. (So it gets skipped where
possible)
All of the internal hooks have been rewritten to use this new mechanism,
and the prescaler hooks have finally been separated from each other.
This also means the prescale FBO kludge is no longer required.
This fixes image corruption for image formats like 0bgr, and also fixes
prescaling under rotation. (As well as other user hooks that have
orientation-dependent access)
The "raw" attributes (tex, tex_pos, pixel_size) are still un-rotated, in
case something needs them, but ideally the hooks should be rewritten to
use the new API as much as possible. The hooked texture has been renamed
from just NAME to NAME_raw to make script authors notice the change (and
also deemphasize direct texture access).
This is also a step towards getting rid of the use_integer pass.
This replaces the previous TRANSFORM by WIDTH, HEIGHT and OFFSET where
WIDTH and HEIGHT are RPN expressions. This allows for more fine-grained
control over the output size, and also makes sure that overwriting
existing textures works more cleanly.
(Also add some more useful bstr functions)
This allows users to add their own near-arbitrary hooks to the vo_opengl
processing pipeline, greatly enhancing the flexibility of user shaders.
This enables, among other things, user shaders such as CrossBilateral,
SuperRes, LumaSharpen and many more.
To make parsing the user shaders easier, shaders are now loaded as
bstrs, and the hooks are set up during video reconfig instead of on
every single frame.
These are "sequence points" where the image could be rendered out to an
FBO, hooked, and re-loaded if any such hook exists. This is perfect for
things like the current user shaders system, as well as optional effects
like unsharp masking.
Note that since we have to pick *some* FBO to store the optionally
hooked texture, we just store it in an array indexed by an increasing
counter. Since we only ever store as many as MAX_TEXTURE_HOOKS + all
internal hook points entries, this is guaranteed to be enough space.
This commit also removes some of the now unused FBOs.
The hook mechanism allows arbitrary processing stages to get dispatched
whenever certain named textures have been "finalized" by the code.
This is mostly meant to serve as a change that opens up the internal
processing in pass_read_video to user scripts, but as a side benefit all
of the code dealing with offsets and plane alignment and other such
confusing things has been rewritten.
This hook mechanism is powerful enough to cover the needs of both
debanding and prescaling (and more), so as a result they can be removed
from pass_read_video entirely and implemented through hooks.
Some avenues for optimization:
- The prescale hook is currently somewhat distributed code-wise. It might be
cleaner to split it into superxbr and NNEDI3 hooks which can each be
self-contained.
- It might be possible to move a large part of the hook code out to an
external file (including the hook definitions for debanding and
prescaling), which would be very much desired.
- Currently, some stages (chroma merging, integer conversion) will
*always* run even if unnecessary. I'm planning another series of
refactors (deferred img_tex) to allow dropping unnecessary shader
stages like these, but that's probably some ways away. In the meantime
it would be doable to re-add some of the logic to skip these stages if
we know we don't need them.
- More hook locations could be added (?)
Instead of rounding down, we round to the nearest float. This reduces
the maximum possible error introduced by this rounding operation. Also
clarify the comment.
Fixes broken colors with --vf=format=0bgr (but only if deband is
disabled).
0bgr means the first byte is padding, while the following three bytes
are bgr. From the vo_opengl perspective, it has 4 physical components
with 3 logical components. copy_img_tex() simply copied 3 components
from the physical representation, which means the last component (r) was
sliced off.
Fix this by not using p->color_swizzle for packed formats, and instead
let packed formats set the per-plane swizzle in texplane.swizzle. The
latter applies the swizzle as part of operation in copy_img_tex(), which
essentially moves physical to logical representations.
Unfortunately, debanding (and thus with opengl-hq defaults) is still
broken.
Make the find_plane_format function take a bit count.
This also makes the function's comment true for the first time the
function and its comment exist. (It was commented as taking bits, but
always took bytes.)
Now that we know in advance whether an implementation should support a
specific format, we have more flexibility when determining which format
to use.
In particular, we can drop the roundabout ES logic.
I'm not sure if actually trying to create the FBO for probing still has
any value. But it might, so leave it for now.
Even if everything else is available, the need for first class arrays
breaks it. In theory we could fix this since we don't strictly need
them, but I guess it's not worth bothering.
Also give the misnamed have_mix variable a slightly better name.
This merges all knowledge about texture format into a central table.
Most of the work done here is actually identifying which formats exactly
are supported by OpenGL(ES) under which circumstances, and keeping this
information in the format table in a somewhat declarative way. (Although
only to the extend needed by mpv.) In particular, ES and float formats
are a horrible mess.
Again this is a big refactor that might cause regression on "obscure"
configurations.
In theory this was needed for the previous commit (but wasn't in
practice, since for hwdec the LUMINANCE_ALPHA mangling is not applied
anymore, and ANGLE uses RG textures in absence of GL_ARB_texture_rg for
whatever crazy reasons).
In practice this caused funky colors on OSX with the uyvy422 format,
which is also fixed in this commit.
Rename gl_hwdec_driver.map_image to map_frame, and let it fill out a
struct gl_hwdec_frame describing the exact texture layout. This gives
more flexibility to what the hwdec interop can export. In particular, it
can export strange component orders/permutations and textures with
padded size. (The latter originating from cropped video.)
The way gl_hwdec_frame works is in the spirit of the rest of the
vo_opengl video processing code, which tends to put as much information
in immediate state (as part of the dataflow), instead of declaring it
globally. To some degree this duplicates the texplane and img_tex
structs, but until we somehow unify those, it's better to give the hwdec
state its own struct. The fact that changing the hwdec struct would
require changes and testing on at least 4 platform/GPU combinations
makes duplicating it almost a requirement to avoid pain later.
Make gl_hwdec_driver.reinit set the new image format and remove the
gl_hwdec.converted_imgfmt field.
Likewise, gl_hwdec.gl_texture_target is replaced with
gl_hwdec_plane.gl_target.
Split out a init_image_desc function from init_format. The latter is not
called in the hwdec case at all anymore. Setting up most of struct
texplane is also completely separate in the hwdec and normal cases.
video.c does not check whether the hwdec "mapped" image format is
supported. This should not really happen anyway, and if it does, the
hwdec interop backend must fail at creation time, so this is not an
issue.
This gives us 16 bit fixed-point integer texture formats, including
ability to sample from them with linear filtering, and using them as FBO
attachments.
The integer texture format path is still there for the sake of ANGLE,
which does not support GL_EXT_texture_norm16 yet.
The change to pass_dither() is needed, because the code path using
GL_R16 for the dither texture relies on glTexImage2D being able to
convert from GL_FLOAT to GL_R16. GLES does not allow this. This could be
trivially fixed by doing the conversion ourselves, but I'm too lazy to
do this now.
The active texture and some pixelstore parameters are now always reset
to defaults when entering and leaving the renderer. Could be important
for libmpv.
Apply basic transformations like rotation by 90° and mirroring when
sampling from the source textures. The original idea was making this
part of img_tex.transform, but this didn't work: lots of code plays
tricks on the transform, so manipulating it is not necessarily
transparent, especially when width/height are switched. So add a new
pre_transform field, which is strictly applied before the normal
transform.
This fixes most glitches involved with rotating the image.
Cropping and rotation are now weirdly separated, even though they could
be done in the same step. I think this is not much of a problem, and
has the advantage that changing panscan does not trigger FBO
reallocations (I think...).
This commit refactors the 3DLUT loading mechanism to build the 3DLUT
against the original source characteristics of the file. This allows us,
among other things, to use a real BT.1886 profile for the source. This
also allows us to actually use perceptual mappings. Finally, this
reduces errors on standard gamut displays (where the previous 3DLUT
target of BT.2020 was unreasonably wide).
This also improves the overall accuracy of the 3DLUT due to eliminating
rounding errors where possible, and allows for more accurate use of
LUT-based ICC profiles.
The current code is somewhat more ugly than necessary, because the idea
was to implement this commit in a working state first, and then maybe
refactor the profile loading mechanism in a later commit.
Fixes#2815.
This also draws it after color management etc. In a nutshell, this
change makes the transparency checkerboard independent of upscaling,
panning, cropping etc. It will always be the same apparent size and
position (relative to the window).
It will also be independent of the video colorspace and such things.
(Note: This might cause white imbalance issues if playing a file with a
white point that does not match the display, in absolute colorimetric
mode. But that's uncommon, especially in conjunction with transparent
image files, so it's not a primary concern here)
Until now, we've let the windowing backend decide. But since they
usually require premultiplied alpha, and premultiplied alpha is easier
to handle, hardcode it.
The recent changes fixed rotation handling, but reversed the rotation
direction. The direction is expected to be counter-clockwise, because
demuxers export video rotation metadata as such.
This has been completely broken since commit 93546f0c. But even before,
rotation handling did not make too much sense. In particular, it rotated
the contents of the cropped image, instead of adjusting the crop
rectangle as well. The result was that things like panscan or zooming
did not behave as expected with rotation applied.
The same is true for vertical flipping. Flipping is triggered by
negative image stride. OpenGL does not support flipping the image on
upload, so it's done as part of the rendering. It can be triggered with
--vf=flip, but other filters and even decoders could setup negative
stride to flip the image.
Fix these issues by applying transforms to texture coordinates properly,
and by making rotation and flipping part of these transforms.
This still doesn't work properly for separated scaling. The issue is
that we'd have to adjust how the passes are done. For now, pick a very
stupid solution by rotating the image to a FBO, and then scaling from
that. This has the avantage that the scale logic doesn't have to be
complicated for such a rare case. It could be improved later.
Prescaling is apparently still broken. I don't know if chroma
positioning works properly either. None of this should affect the case
with no rotation.
If the texture count is lower than 4, entries in va.textcoord[] will
remain uninitialized. While this is unlikely to be a problem (since
these values are unused on the shader side too), it's not nice and might
explain some things which have shown up in valgrind.
Fix by always initializing the whole thing.
Glitches when resizing are still possible, but are reduced. Other VOs
could support this too, but don't need to do so.
(Totally avoiding glitches would be much more effort, and probably not
worth the trouble. How about you just watch the video the player is
playing, instead of spending your time resizing the window.)
This also gets rid of the kind of hard to read texture swizzle setup and
turns it into something dumber.
Assumes that we don't create any FBOs with 2 channel formats. (Only the
video source textures are handled by this commit.)
This is a fresh implementation from scratch that carries with it
significantly less baggage and verbosity from the previous (ported)
version.
The actual values for the masks and such were copied from the
current code. Behavior and performance should be unaffected.
An important difference between the old code and the new code is that
the new code always explicitly samples from the first component, rather
than being able to process multiple planes at once.
Since prescale-luma only affects luma, I deemed this unnecessary. May
change in the future, if prescale-chroma ever gets implemented. But
prescaling multiple planes would be slow to do this way. (Better would
be to generalize it to differently-sized vectors)
Instead of hard-coding the logic and planes to skip, factor this out
to a reusible function, and instead add the number of relevant
coordinates to the texture state.
Since prescale now literally only affects the luma plane (and the
filters are all designed for luma-only operation either way), the option
has been renamed and the documentation updated to clarify this.
This is a pretty major rewrite of the internal texture binding
mechanic, which makes it more flexible.
In general, the difference between the old and current approaches is
that now, all texture description is held in a struct img_tex and only
explicitly bound with pass_bind. (Once bound, a texture unit is assumed
to be set in stone and no longer tied to the img_tex)
This approach makes the code inside pass_read_video significantly more
flexible and cuts down on the number of weird special cases and
spaghetti logic.
It also has some improvements, e.g. cutting down greatly on the number
of unnecessary conversion passes inside pass_read_video (which was
previously mostly done to cope with the fact that the alternative would
have resulted in a combinatorial explosion of code complexity).
Some other notable changes (and potential improvements):
- texture expansion is now *always* handled in pass_read_video, and the
colormatrix never does this anymore. (Which means the code could
probably be removed from the colormatrix generation logic, modulo some
other VOs)
- struct fbo_tex now stores both its "physical" and "logical"
(configured) size, which cuts down on the amount of width/height
baggage on some function calls
- vo_opengl can now technically support textures with different bit
depths (e.g. 10 bit luma, 8 bit chroma) - but the APIs it queries
inside img_format.c doesn't export this (nor does ffmpeg support it,
really) so the status quo of using the same tex_mul for all planes is
kept.
- dumb_mode is now only needed because of the indirect_fbo being in the
main rendering pipeline. If we reintroduce p->use_indirect and thread
a transform through the entire program this could be skipped where
unnecessary, allowing for the removal of dumb_mode. But I'm not sure
how to do this in a clean way. (Which is part of why it got introduced
to begin with)
- It would be trivial to resurrect source-shader now (it would just be
one extra 'if' inside pass_read_video).
Why was this done so stupidly, with so many complicated special cases,
before? Declare it once so the shader bits don't have to figure out where
and when to do so themselves.
Apple crap (namely hardware decoding interop) forces us to use rectangle
textures for input. But after that we continue with normal textures.
This was not considered for debanding, and the sampler type used for it
can be different depending on the exact render chain. Simply use the
target type of the input texture.
It thinks that integer_conv_fbo[index] is implied to be accessed with up
to index=5. Although that is theoretical only, it has a point that this
makes no sense. Use the same constant for the array allocation, to make
it more uniform and robust.
Fixes CID 1350060.
GLES does not support high bit depth fixed point textures for unknown
reasons, so direct 10 bit input is not possible. But we can still use
integer textures, which are supported by GLES 3.0. These store integer
data just like the standard fixed point textures, except they are not
normalized on sampling. They also don't support bilinear filtering, and
require a special sampler ("usampler2D").
While these texture formats enable us to shuffle the data to the GPU,
they're rather impractical with the requirements mentioned above and our
current architecture. One problem is that most code assumes it can
always use bilinear scaling (even if bilinear is never used when using
appropriate scale/cscale options). Another is that we don't have any
concept of running a function on a texture in an uniform way.
So for now, run a simple conversion step through a FBO. The FBO will use
the rgba16f format normally, which gives enough bits for 10 bit, and
will at least gracefully degrade with higher depth input.
This is bound to be much slower than a more "direct" method, but at
least it works and is simple to implement.
The odd change of function call order in init_video() is to properly
disable "dumb mode" (no FBO use) if these texture formats are in use.
This was never reset - absolutely can't be right. If the renderer
somehow switches back to another codepath, it certainly has to be reset.
Maybe this was hard to hit, as the normalization is going to be
idempotent in simpler cases (like rendering RGBA input).
Also get rid of the "merged" variable.
