video: handle endian detection in a more generic way

FFmpeg has only a AV_PIX_FMT_FLAG_BE flag, not a LE one, which causes
problems for us: we want to have the LE flag too, so code can actually
detect whether a format is non-native endian. Basically, we want to
reconstruct the LE/BE suffix all AV_PIX_FMT_*s have.

Doing this is hard due to the (messed up) way AVPixFmtDescriptor works.
The worst is AV_PIX_FMT_RGB444: this group of formats describe an
endian-independent access (since no component actually spans 2 bytes,
you only need byte accesses with a fixed offset), so we have to go
through some pain.
This commit is contained in:
wm4 2014-11-04 23:32:02 +01:00
parent 3a50bbf563
commit 9548f63943
1 changed files with 21 additions and 7 deletions

View File

@ -144,12 +144,14 @@ struct mp_imgfmt_desc mp_imgfmt_get_desc(int mpfmt)
int planedepth[4] = {0};
int el_size = (pd->flags & AV_PIX_FMT_FLAG_BITSTREAM) ? 1 : 8;
bool need_endian = false; // single component is spread over >1 bytes
for (int c = 0; c < pd->nb_components; c++) {
AVComponentDescriptor d = pd->comp[c];
// multiple components per plane -> Y is definitive, ignore chroma
if (!desc.bpp[d.plane])
desc.bpp[d.plane] = (d.step_minus1 + 1) * el_size;
planedepth[d.plane] += d.depth_minus1 + 1;
need_endian |= (d.depth_minus1 + 1 + d.shift) > 8;
}
for (int p = 0; p < 4; p++) {
@ -157,19 +159,31 @@ struct mp_imgfmt_desc mp_imgfmt_get_desc(int mpfmt)
desc.num_planes++;
}
// Packed RGB formats are the only formats that have less than 8 bits per
// component, and still require endian dependent access.
if (pd->comp[0].depth_minus1 + 1 <= 8 &&
!(mpfmt >= IMGFMT_RGB444_LE && mpfmt <= IMGFMT_BGR565_BE))
{
desc.plane_bits = planedepth[0];
// Check whether any components overlap other components (per plane).
// We're cheating/simplifying here: we assume that this happens if a shift
// is set - which is wrong in general (could be needed for padding, instead
// of overlapping bits of another component). Needed for rgb444le/be.
bool component_byte_overlap = false;
for (int c = 0; c < pd->nb_components; c++) {
AVComponentDescriptor d = pd->comp[c];
component_byte_overlap |= d.shift > 0 && planedepth[d.plane] > 8;
}
// If every component sits in its own byte, or all components are within
// a single byte, no endian-dependent access is needed. If components
// stride bytes (like with packed 2 byte RGB formats), endian-dependent
// access is needed.
need_endian |= component_byte_overlap;
if (!need_endian) {
desc.flags |= MP_IMGFLAG_LE | MP_IMGFLAG_BE;
} else {
desc.flags |= (pd->flags & AV_PIX_FMT_FLAG_BE)
? MP_IMGFLAG_BE : MP_IMGFLAG_LE;
}
desc.plane_bits = planedepth[0];
if (mpfmt == IMGFMT_XYZ12_LE || mpfmt == IMGFMT_XYZ12_BE) {
desc.flags |= MP_IMGFLAG_XYZ;
} else if (!(pd->flags & AV_PIX_FMT_FLAG_RGB) &&