vo_gpu: hwdec_vaapi: Count planes rather than layers in Vulkan interop

We saw a segfault when trying to use the intel-media-driver (iHD)
rather than the normal intel va driver. This happened because the
iHD driver reports P010 (and maybe other formats) with multiple
layers to represent the interleaved UV plane. The normal va driver
reports one UV layer to match the plane.

This threw off my logic which assumed that the number of layers
could not exceed the number of planes.

There's a way one could fix this in a fully generalised form, but
I'm just going to do what the EGL path does and assume that:
 * Layer 'n' is on Plane 'n' for n < total number of planes
 * These layers always start at offset 0 on the plane

You can imagine ways that these assumptions are violated, but at
least the failure will look the same for both EGL and Vulkan
paths.
This commit is contained in:
Philip Langdale 2019-06-22 10:38:44 -07:00 committed by sfan5
parent b33ced193e
commit b5b0350371
1 changed files with 1 additions and 1 deletions

View File

@ -472,7 +472,7 @@ static int mapper_map(struct ra_hwdec_mapper *mapper)
if (!ra_get_imgfmt_desc(mapper->ra, mapper->dst_params.imgfmt, &desc))
goto esh_failed;
for (int n = 0; n < p->desc.num_layers; n++) {
for (int n = 0; n < p->num_planes; n++) {
if (p->desc.layers[n].num_planes > 1) {
// Should never happen because we request separate layers
MP_ERR(mapper, "Multi-plane VA surfaces are not supported\n");