[VDPAU] ffmpeg and HEVC: NumLongTermPicturesSlcieHeaderBits

[Jose Soltren]

Hello Philip, please see my reply inline below.

On 2015/05/22, 17:16 , "Philip Langdale" <philipl at overt.org> wrote:

>Hi all,
>
>I've started working on exposing the VDPAU HEVC functionality in ffmpeg,
>and this means
>mapping all the metadata from ffmpeg structures to VDPAU. I've managed
>to, I believe,
>map most of the fields at this point - some were obvious, some I could
>borrow from the
>dxva equivalent code and one time, there was a helper function inside
>ffmpeg that did the
>necessary calculation, but this one has me stumped:
>
>     /** Second, VDPAU requires the number of bits used for long term
>reference
>         pictures in the slice_segment_header. This is equal to the
>number
>         of bits used for the contents of the block beginning with
>         "if(long_term_ref_pics_present_flag)". */
>     uint32_t NumLongTermPictureSliceHeaderBits;
>
>I've found the code in ffmpeg that does the calculation described in the
>spec, but I still
>don't understand how to get from there to this bit count. Is this
>supposed to reflect how
>many bits are extracted and read by the code fragment?

Sort of.

Recall that H.265/HEVC is written from the ground up for explicit
parallelism. A picture can be divided into slices that are decoded
independently. This includes independence at the bitstream level which
leads to some redundant amount of data across slice headers.

Some hardware implementations (like NVIDIA's!) may choose instead to
perform some amount of bitstream parsing up front before decoding
individual slices. The decoder needs to know the "number of bits to skip"
for all following frames. This is what you are calculating here.

The exact specification for this number of bits is available in the
description of the slice segment header in the Specification. This was
7.3.6.1 General slice segment header syntax in Rec. ITU-T H.265 (04/2013).
Things may have moved around a bit in later Specifications.

At some point all slice segment header parsers start to look pretty
similar regardless of who is writing them since they all need to comply to
the Specification.

Locally, I hacked some gstreamer code to get this working.

NumLongTermPictureSliceHeaderBits must be equal to the actual number of
bits (not bytes, bits) used by all of the structures that are only present
in the slice segment header if long_term_ref_pics_present_flag is set in
the SPS associated with this picture.


The same is true for NumShortTermPictureSliceHeaderBits: it is the number
of bits of everything after short_term_ref_pic_set_sps_flag, and before
and not including the if(long_term_ref_pics_present_flag) line.

Both of these can be variable length. So, it makes sense to piggyback onto
whatever is already counting bits.

(If it is any consolation I spent a very long time staring at 7.3.6.1,
making diagrams, and asking questions.)

>In other words, 
>would I need to modify
>the ffmpeg code to track how many bits it's reading as it goes and then
>expose that total to
>me to pass on?

Yes.

>This bit count concept isn't used by dxva or by the
>software decoder so I
>don't have anything to compare with.

Indeed. Ideally this would be part of DXVA, or even better, the H.265/HEVC
Specification. (I wonder if Gary Sullivan reads this list.)

>ffmpeg code is here:
>http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavcodec/hevc.c;h=df6cce6
>f8a2cf5dba6bfb1e3bc258462a6b97a3f;hb=HEAD#l231

IIRC a GetBitContext will keep an index value that tells you its current
position. This could be useful here.

I would compare decode_lt_rps() to the chart in 7.3.6.1 as mentioned
above, see exactly what structures decode_lt_rps() is decoding, and get
the bit position out of gb->index before and after these are parsed. Then,
take the difference, and pass that value to VDPAU as
NumLongTermPictureSliceHeaderBits.

Thanks,
--José

nvpublic

>Thanks,
>
>--phil
>
>_______________________________________________
>VDPAU mailing list
>VDPAU at lists.freedesktop.org
>http://lists.freedesktop.org/mailman/listinfo/vdpau