lavc/vvc: Fix check whether QG is in first tile col

The second part of this condition is intended to check whether the
current quantisation group is in the first CTU column of the current
tile.  The issue is that ctb_to_col_bd gives the x-ordinate of the first
column of the current tile *in CTUs*, while xQg gives the x-ordinate of
the quantisation group *in samples*.  Rectify this by shifting xQg by
ctb_log2_size to get xQg in CTUs before comparing.

Fixes FFVVC issues #201 and #203.
This commit is contained in:
Frank Plowman 2024-03-19 19:47:51 +00:00 committed by Nuo Mi
parent 0b7d4fccce
commit dfcf5f828d

View File

@ -96,7 +96,7 @@ static int get_qp_y_pred(const VVCLocalContext *lc)
if (lc->na.cand_up) {
const int first_qg_in_ctu = !(xQg & ctb_size_mask) && !(yQg & ctb_size_mask);
const int qPy_up = fc->tab.qp[LUMA][x_cb + (y_cb - 1) * min_cb_width];
if (first_qg_in_ctu && pps->ctb_to_col_bd[xQg >> ctb_log2_size] == xQg)
if (first_qg_in_ctu && pps->ctb_to_col_bd[xQg >> ctb_log2_size] == xQg >> ctb_log2_size)
return qPy_up;
}