1

The HEVC quantization(Uniform case) step in HEVC standard requires right shifting by QP/6 while calculating the level coefficients.

I am not sure how to perform this right shifting for cases when QP is not divisble by 6. Any help would be appreciated.

Reference: Core Transform Design in the High Efficiency Video Coding (HEVC) Standard: Madhukar Budagavi, Senior Member, IEEE, Arild Fuldseth, Gisle Bjøntegaard, Vivienne Sze, Member, IEEE, and Mangesh Sadafale

damjeux
  • 320
  • 2
  • 12
Sourav Sen
  • 11
  • 4

1 Answers1

5

In the HEVC reference software (HM 16.5), the quantization step q is calculated as follows:

Double q[MAX_QP + 1];
for (int v = 0; v <= MAX_QP; v++)
{
    q[v] = (Double)(g_invQuantScales[v % 6] * (1 << (v / 6))) / 64;
}

where g_invQuantScales is defined as follows:

const Int g_invQuantScales[SCALING_LIST_REM_NUM] =
{
  40,45,51,57,64,72
};

That is, as v is an integer, v/6 is an integer division. The behavior of integer division in c is explained for example here: What is the behavior of integer division in C?

Basically, as v is always non-negative here, the result of the integer division will be the same as the floor operation on the exact quotient.

In that sense, equation (8) in the paper you referenced is a bit misleading, as the writing is in "code style".

If you need to implement it yourself in another language, you should make sure that you use a floor operation. E.g. " << floor( QP/6) "

damjeux
  • 320
  • 2
  • 12