[VDPAU] [PATCH v6] Extend the VDPAU API to support H.265/HEVC Decoding
José Hiram Soltren
jsoltren at nvidia.com
Tue Feb 3 08:36:15 PST 2015
Hi Aaron,
Comments inline below.
On 02/02/2015 06:07 PM, Aaron Plattner wrote:
> On 01/29/2015 02:36 PM, José Hiram Soltren wrote:
>> This patch adds an API for player applications to utilize VDPAU for
>> hardware-accelerated playback of H.265/HEVC streams.
>>
>> The goals of this API are:
>> - enable hardware accelerated decoding of H.265/HEVC content under VDPAU;
>> - provide a reference implementation for H.265/HEVC hardware decoding that
>> is vendor agnostic;
>> - provide enough data for H.265/HEVC hardware acceleration implementations
>> from multiple vendors to be able to use the same API;
>>
>> This patch is written against "version one" of the H.265/HEVC Specification,
>> Rec. ITU-T H.265 (04/2013), available at:
>>
>> http://handle.itu.int/11.1002/1000/12296
>>
>> A future patch against this header may address bug fixes, and may support
>> the new features described in "version two" of the H.265/HEVC Specification,
>> Rec. ITU-T H.265 v2 (10/2014).
>>
>> Note that the API does need to be self documenting with Doxygen markup,
>> which we (NVIDIA) will generate and post as an update to our public VDPAU
>> documentation.
>>
>> This is version 6 of this patch.
>>
>> Version 1 was the original version.
>>
>> Version 2 was a minor cleanup change.
>>
>> Version 3 incorporated 10- and 12-bit formats.
>>
>> Version 4 clarified some documentation related to H.265/HEVC support.
>>
>> Version 5 clarified some documentation related to H.265/HEVC support
>> and correcting the Specification URI above.
>>
>> Version 6 further corrected the Specification URI above, re-ordered the
>> fields in VdpPictureInfoHEVC to agree with the Specification, and added
>> additional documentation for some fields. It also corrected some cosmetic
>> indentation errors.
>
> Thanks for this. This helps correlate it to section 7.3.2, "Raw byte sequence
> payloads, trailing bits, and byte alignment syntax".
>
> Sorry about all the noob questions below.
(If it's any consolation I've had to answer the same questions myself through
the development of this API. There are a whole bunch of fiddly little things to
get right for this codec.)
>> ---
>> include/vdpau/vdpau.h | 314 ++++++++++++++++++++++++++++++++++++++++++++++++-
>> 1 file changed, 313 insertions(+), 1 deletion(-)
>>
>> diff --git a/include/vdpau/vdpau.h b/include/vdpau/vdpau.h
>> index b386825..e3b1c68 100644
>> --- a/include/vdpau/vdpau.h
>> +++ b/include/vdpau/vdpau.h
>> @@ -6,7 +6,7 @@
>> /*
>> * This copyright notice applies to this header file:
>> *
>> - * Copyright (c) 2008-2010 NVIDIA Corporation
>> + * Copyright (c) 2008-2015 NVIDIA Corporation
>> *
>> * Permission is hereby granted, free of charge, to any person
>> * obtaining a copy of this software and associated documentation
>> @@ -432,6 +432,26 @@
>> * Include all slices beginning with start codes 0x000001B6. The slice start
>> * code must be included for all slices.
>> *
>> + * \subsection bitstream_hevc H.265/HEVC - High Efficiency Video Codec
>> + *
>> + * Include all video coding layer (VCL) NAL units, with nal_unit_type values
>> + * of 0 (TRAIL_N) through 31 (RSV_VCL31) inclusive. In addition to parsing
>> + * and providing NAL units, an H.265/HEVC decoder application using VDPAU
>> + * for decoding must parse certain values of the first slice segment header
>> + * in a VCL NAL unit and provide it through VdpPictureInfoHEVC. Please see
>> + * the documentation for VdpPictureInfoHEVC below for further details.
>> + *
>> + * The complete slice start code (including the 0x000001 prefix) must be
>> + * included for all slices, even when the prefix is not included in the
>> + * bitstream.
>> + *
>> + * Note that if desired:
>> + *
>> + * - The slice start code prefix may be included in a separate bitstream
>> + * buffer array entry to the actual slice data extracted from the bitstream.
>> + * - Multiple bitstream buffer array entries (e.g. one per slice) may point at
>> + * the same physical data storage for the slice start code prefix.
>> + *
>> * \section video_mixer_usage Video Mixer Usage
>> *
>> * \subsection video_surface_content VdpVideoSurface Content
>> @@ -2463,6 +2483,17 @@ typedef uint32_t VdpDecoderProfile;
>> /** \hideinitializer */
>> /** \brief Support for 8 bit depth only */
>> #define VDP_DECODER_PROFILE_H264_HIGH_444_PREDICTIVE
>> ((VdpDecoderProfile)26)
>> +/** \hideinitializer */
>> +/** \brief MPEG-H Part 2 == H.265 == HEVC */
>> +#define VDP_DECODER_PROFILE_HEVC_MAIN ((VdpDecoderProfile)50)
>> +/** \hideinitializer */
>> +#define VDP_DECODER_PROFILE_HEVC_MAIN_10 ((VdpDecoderProfile)51)
>> +/** \hideinitializer */
>> +#define VDP_DECODER_PROFILE_HEVC_MAIN_STILL ((VdpDecoderProfile)52)
>> +/** \hideinitializer */
>> +#define VDP_DECODER_PROFILE_HEVC_MAIN_12 ((VdpDecoderProfile)53)
>> +/** \hideinitializer */
>> +#define VDP_DECODER_PROFILE_HEVC_MAIN_444 ((VdpDecoderProfile)54)
>>
>> /** \hideinitializer */
>> #define VDP_DECODER_LEVEL_MPEG1_NA 0
>> @@ -2558,6 +2589,38 @@ typedef uint32_t VdpDecoderProfile;
>> #define VDP_DECODER_LEVEL_DIVX_NA 0
>>
>> /**
>> + * The VDPAU H.265/HEVC decoder levels correspond to the values of
>> + * general_level_idc as described in the H.265 Specification, Annex A,
>> + * Table A.1. The enumeration values are equal to thirty times the level
>> + * number.
>> + */
>> +#define VDP_DECODER_LEVEL_HEVC_1 30
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_2 60
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_2_1 63
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_3 90
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_3_1 93
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_4 120
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_4_1 123
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_5 150
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_5_1 153
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_5_2 156
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_6 180
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_6_1 183
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_6_2 186
>> +
>> +/**
>> * \brief Query the implementation's VdpDecoder capabilities.
>> * \param[in] device The device to query.
>> * \param[in] profile The decoder profile for which information is requested.
>> @@ -3054,6 +3117,255 @@ typedef VdpPictureInfoMPEG4Part2 VdpPictureInfoDivX4;
>> typedef VdpPictureInfoMPEG4Part2 VdpPictureInfoDivX5;
>>
>> /**
>> + * \brief Picture parameter information for an H.265/HEVC picture.
>> + *
>> + * Note: References to "copy of ... bitstream field" in the field
>> + * descriptions may refer to data literally parsed from the bitstream,
>> + * or derived from the bitstream using a mechanism described in
>> + * Rec. ITU-T H.265 (04/2013), hereafter referred to as
>> + * "the H.265/HEVC Specification".
>> + *
>> + * Client applications must supply every field in this struct.
>> + */
>> +typedef struct {
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t chroma_format_idc;
>> + /** Copy of the Sequence Parameter Set bitstream field.
>> + Only valid if chroma_format_idc == 3. */
>> + uint8_t separate_colour_plane_flag;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint32_t pic_width_in_luma_samples;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint32_t pic_height_in_luma_samples;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t bit_depth_luma_minus8;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t bit_depth_chroma_minus8;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t log2_max_pic_order_cnt_lsb_minus4;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t sps_sub_layer_ordering_info_present_flag;
>
> I was confused by this so I asked off-list. To summarize the response, in the
> bitstream, the next field is conditional on this flag. If it's true, it's an
> array indexed by the temporal_ids. So a player needs to choose the correct
> field in the array based on this flag sps_max_sub_layers_minus1, and the
> temporal_id of the current NAL unit and copy that value into
> sps_max_dec_pic_buffering_minus1 below. But an implementation doesn't actually
> need sps_sub_layer_ordering_info_present_flag.
Yes. The only use case of sps_sub_layer_ordering_info_present_flag in the
Specification is to help select correct values of
sps_max_dec_pic_buffering_minus1[i], sps_max_num_reorder_pics[i], and
sps_max_latency_increase_plus1[i]. Since the actual user of the
sps_sub_layer_ordering_info_present_flag would be the parsing process for the
sequence parameter set RBSP syntax, only, there is no need to pass it along to
VDPAU.
Note that VDPAU implementations against this header basically implement clauses
8.3 through 8.7 of the Specification.
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t sps_max_dec_pic_buffering_minus1;
>
> It might help to add a comment mentioning that it's the appropriate value from
> the array for the current NAL's temporal_id.
Yes. I will add that to the comments.
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t log2_min_luma_coding_block_size_minus3;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t log2_diff_max_min_luma_coding_block_size;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t log2_min_transform_block_size_minus2;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t log2_diff_max_min_transform_block_size;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t max_transform_hierarchy_depth_inter;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t max_transform_hierarchy_depth_intra;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t scaling_list_enabled_flag;
>
> I assume if this is true, it's expected for the player to fill in the scaling
> parameters specified in the PPS, the SPS, or the built-in defaults specified in
> section 7.4.5 into the "scaling lists (diag order)" section below, right?
Yes. If it would be clearer I can move the scaling lists up here. In the
sequence parameter set RBSP syntax, scaling_list_data() immediately follows
sps_scaling_list_data_present_flag and scaling_list_enabled_flag.
More concretely: the scaling list values in VdpPictureInfoHEVC are the values
that the (effectively stateless) VDPAU implementation will use immediately to
decode the raw bitstream data passed to VdpDecoderRender. It is up to the
player to follow the Specification and provide the correct scaling lists.
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t amp_enabled_flag;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t sample_adaptive_offset_enabled_flag;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t pcm_enabled_flag;
>> + /** Copy of the Sequence Parameter Set bitstream field.
>> + Only needs to be set if pcm_enabled_flag is set. */
>> + uint8_t pcm_sample_bit_depth_luma_minus1;
>> + /** Copy of the Sequence Parameter Set bitstream field.
>> + Only needs to be set if pcm_enabled_flag is set. */
>> + uint8_t pcm_sample_bit_depth_chroma_minus1;
>> + /** Copy of the Sequence Parameter Set bitstream field.
>> + Only needs to be set if pcm_enabled_flag is set. */
>> + uint8_t log2_min_pcm_luma_coding_block_size_minus3;
>> + /** Copy of the Sequence Parameter Set bitstream field.
>> + Only needs to be set if pcm_enabled_flag is set. */
>> + uint8_t log2_diff_max_min_pcm_luma_coding_block_size;
>> + /** Copy of the Sequence Parameter Set bitstream field.
>> + Only needs to be set if pcm_enabled_flag is set. */
>> + uint8_t pcm_loop_filter_disabled_flag;
>> + /** Copy of the Sequence Parameter Set bitstream field.
>> + Per spec, when zero, assume short_term_ref_pic_set_sps_flag
>> + is also zero. */
>> + uint8_t num_short_term_ref_pic_sets;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t long_term_ref_pics_present_flag;
>> + /** Copy of the Sequence Parameter Set bitstream field.
>> + Only needed if long_term_ref_pics_present_flag is set. */
>> + uint8_t num_long_term_ref_pics_sps;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t sps_temporal_mvp_enabled_flag;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t strong_intra_smoothing_enabled_flag;
>
> I assume the "vui_parameters" and "sps_extension" stuff aren't useful.
The VUI parameters - video usability information - correspond to data that a
player application would need to display and present a video correctly, not to
decode it. VUI is covered in Annex E of the Specification. From Annex E: "VUI
parameters are not required for constructing the luma or chroma samples by the
decoding process."
The player may choose to parse and interpret the VUI - and likely should - but
as the VDPAU implementation implements clauses 8.3 through 8.7 it would not
need this information.
sps_extension is not defined in this version (04/2013) of the Specification and
is reserved for future use. We can safely ignore it. See sps_extension_flag and
sps_extension_data_flag at the end of 7.4.3.2.
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t dependent_slice_segments_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t output_flag_present_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t num_extra_slice_header_bits;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t sign_data_hiding_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t cabac_init_present_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t num_ref_idx_l0_default_active_minus1;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t num_ref_idx_l1_default_active_minus1;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + int8_t init_qp_minus26;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t constrained_intra_pred_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t transform_skip_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t cu_qp_delta_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only needed if cu_qp_delta_enabled_flag is set. */
>> + uint8_t diff_cu_qp_delta_depth;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + int8_t pps_cb_qp_offset;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + int8_t pps_cr_qp_offset;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t pps_slice_chroma_qp_offsets_present_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t weighted_pred_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t weighted_bipred_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t transquant_bypass_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t tiles_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t entropy_coding_sync_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only valid if tiles_enabled_flag is set. */
>> + uint8_t num_tile_columns_minus1;
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only valid if tiles_enabled_flag is set. */
>> + uint8_t num_tile_rows_minus1;
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only valid if tiles_enabled_flag is set. */
>> + uint8_t uniform_spacing_flag;
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only need to set 0..num_tile_columns_minus1. The struct
>> + definition reserves up to the maximum of 22. */
>> + uint16_t column_width_minus1[22];
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only need to set 0..num_tile_rows_minus1. The struct
>> + definition reserves up to the maximum of 20. */
>> + uint16_t row_height_minus1[20];
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only needed if tiles_enabled_flag is set. */
>> + uint8_t loop_filter_across_tiles_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t pps_loop_filter_across_slices_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t deblocking_filter_control_present_flag;
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only valid if deblocking_filter_control_present_flag is set. */
>> + uint8_t deblocking_filter_override_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only valid if deblocking_filter_control_present_flag is set. */
>> + uint8_t pps_deblocking_filter_disabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only valid if deblocking_filter_control_present_flag is set. */
>> + int8_t pps_beta_offset_div2;
>
> This is only valid if deblocking_filter_control_present_flag is set *and*
> pps_deblocking_filter_disabled_flag is false, right?
>
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only valid if deblocking_filter_control_present_flag is set. */
>> + int8_t pps_tc_offset_div2;
>
> same here
Yes and yes, as per 7.3.2.3. These comments should read, "only valid if
deblocking_filter_control_present flag is set and
pps_deblocking_filter_disabled_flag is not set".
I wonder how much value these comments add as they are entirely redundant to
the Specification. I will leave them for now.
>> + /** scaling lists (diag order) */
>> + /** Scaling List for 4x4 quantization matrix,
>> + indexed as ScalingList4x4[matrixId][i]. */
>> + uint8_t ScalingList4x4[6][16];
>> + /** Scaling List for 8x8 quantization matrix,
>> + indexed as ScalingList8x8[matrixId][i]. */
>> + uint8_t ScalingList8x8[6][64];
>> + /** Scaling List for 16x16 quantization matrix,
>> + indexed as ScalingList16x16[matrixId][i]. */
>> + uint8_t ScalingList16x16[6][64];
>> + /** Scaling List for 32x32 quantization matrix,
>> + indexed as ScalingList32x32[matrixId][i]. */
>> + uint8_t ScalingList32x32[2][64];
>> + /** Scaling List DC Coefficients for 16x16,
>> + indexed as ScalingListDCCoeff16x16[matrixId]. */
>> + uint8_t ScalingListDCCoeff16x16[6];
>> + /** Scaling List DC Coefficients for 32x32,
>> + indexed as ScalingListDCCoeff32x32[matrixId]. */
>> + uint8_t ScalingListDCCoeff32x32[2];
>
> I'm going to assume all of these scaling list arrays make sense.
Scaling list parsing is described in 7.4.5. There is a default scaling list
that a player is required to maintain. The scaling_list_data() syntax structure
may provide actual values, deltas to reference values, or no values at all.
Table 7-3 maps sizes of quantization matrix values to sizeId values. It is
correct for all but 32x32 to have 2 matrix sets and all others to have 6 as per
line 3 of 7.3.4. The i values go from 0 to coefNum as described also in 7.3.4,
with "coefNum = Min(64, (1<<(4+(sizeId<<1)))). That works out to 16 for 4x4 and
64 for all else.
scaling_list_dc_coef_minus8[][] is only specified for 16x16 and 32x32 (sizeId >
1). Note that this is a matrix of the actual values -
scaling_list_dc_coef_minus8[][] + 8.
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t lists_modification_present_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t log2_parallel_merge_level_minus2;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t slice_segment_header_extension_present_flag;
>> +
>> + /** Slice Segment Header bitstream fields and calculated values. */
>> + /** Set to 1 if nal_unit_type is equal to IDR_W_RADL or IDR_N_LP.
>> + Set to zero otherwise. */
>> + uint32_t IDRPicFlag;
>> + /** Set to 1 if nal_unit_type in the range of BLA_W_LP to
>> + RSV_IRAP_VCL23, inclusive. Set to zero otherwise.*/
>> + uint32_t RAPPicFlag;
>> + /** Copy of specification field, CurrRpsIdx, see 7.4.7.1. */
>> + uint32_t CurrRpsIdx;
>> + /** Copy of specification field, see Section 7.4.7.2 of the
>> + H.265/HEVC Specification. */
>> + uint32_t NumPocTotalCurr;
>> + /** Corresponds to specification field, NumDeltaPocs[RefRpsIdx].
>> + Only applicable when short_term_ref_pic_set_sps_flag == 0.
>> + Implementations will ignore this value in other cases. See 7.4.8. */
>> + uint32_t NumDeltaPocsOfRefRpsIdx;
>> + /** Section 7.6.3.1 of the H.265/HEVC Specification defines the syntax of
>> + the slice_segment_header. This header contains information that
>> + some VDPAU implementations may choose to skip. The VDPAU API
>> + requires client applications to track the number of bits used in the
>> + slice header for structures associated with short term and long term
>> + reference pictures. First, VDPAU requires the number of bits used by
>> + the short_term_ref_pic_set array in the slice_segment_header. */
>> + uint32_t NumShortTermPictureSliceHeaderBits;
>> + /** Second, VDPAU requires the number of bits used for long term reference
>> + pictures in the slice_segment_header. This is equal to the number
>> + of bits used for the contents of the block beginning with
>> + "if(long_term_ref_pics_present_flag)". */
>> + uint32_t NumLongTermPictureSliceHeaderBits;
>> +
>> + /** Slice Decoding Process - Picture Order Count */
>> + /** The value of PicOrderCntVal of the picture in the access unit
>> + containing the SEI message. The picture being decoded. */
>> + int32_t CurrPicOrderCntVal;
>
> These all seem reasonable. Thanks for the comments describing them.
>
> What does it mean for CurrPicOrderCntVal to be negative?
It means that this picture is decoded after an IDR picture ("I-frame") but
displayed before it.
Clause 8.3.1 specifies:
The value of PicOrderCntVal shall be in the range of −2^^31 to 2^^31 − 1,
inclusive. In one CVS, the PicOrderCntVal values for any two coded pictures
shall not be the same.
Consider the comment Note 1 in clause 8.3.1:
NOTE 1 – All IDR pictures will have PicOrderCntVal equal to 0 since
slice_pic_order_cnt_lsb is inferred to be 0 for IDR pictures and
prevPicOrderCntLsb and prevPicOrderCntMsb are both set equal to 0.
Allowing PicOrderCntVal to go negative allows the naming of pictures that come
after an IDR picture in decoding order, but before that same IDR picture in
display order.
>> + /** Slice Decoding Process - Reference Picture Sets */
>> + /** Array of video reference surfaces.
>> + Set any unused positions to VDP_INVALID_HANDLE. */
>> + VdpVideoSurface RefPics[16];
>> + /** Array of picture order counts. These correspond to positions
>> + in the RefPics array. */
>> + int32_t PicOrderCntVal[16];
>> + /** Array used to specify whether a particular RefPic is
>> + a long term reference. A value of "1" indicates a long-term
>> + reference. */
>> + uint8_t IsLongTerm[16];
>> + /** Copy of specification field, see Section 8.3.2 of the
>> + H.265/HEVC Specification. */
>> + uint32_t NumPocStCurrBefore;
>> + /** Copy of specification field, see Section 8.3.2 of the
>> + H.265/HEVC Specification. */
>> + uint32_t NumPocStCurrAfter;
>> + /** Copy of specification field, see Section 8.3.2 of the
>> + H.265/HEVC Specification. */
>> + uint32_t NumPocLtCurr;
>> + /** Reference Picture Set list, one of the short-term RPS. These
>> + correspond to positions in the RefPics array. */
>> + int8_t RefPicSetStCurrBefore[8];
>> + /** Reference Picture Set list, one of the short-term RPS. These
>> + correspond to positions in the RefPics array. */
>> + int8_t RefPicSetStCurrAfter[8];
>> + /** Reference Picture Set list, one of the long-term RPS. These
>> + correspond to positions in the RefPics array. */
>> + int8_t RefPicSetLtCurr[8];
>
> It's not clear to me why there are only 8 of these if RefPics has 16 entries
> and these correspond to entries in that array. How is this correspondence
> defined, exactly?
This is a many to many mapping described in detail in clause 8.3.2, "Decoding
process for reference picture set".
The Specification is somewhat opaque about this in that the answer to this
question lies both here and in Annex C. But, it comes down to this:
- the decoded picture buffer (DPB) may be up to 16 pictures long;
- there are five separate reference picture lists, of which three are actually
active for any picture;
- each individual list may have up to eight elements.
8.3.2 goes on to describe bitstream conformance conditions that imply that the
same picture cannot appear in multiple lists.
>> +} VdpPictureInfoHEVC;
>> +
>> +/**
>> * \brief Decode a compressed field/frame and render the result
>> * into a \ref VdpVideoSurface "VdpVideoSurface".
>> * \param[in] decoder The decoder object that will perform the
>>
>
> This is going to need a corresponding decoder in vdpau_trace.cpp. It doesn't
> have to be part of the same change, but I'd like to merge them together. I can
> try to write that, but I can't promise when I'll have time to do that.
I don't mind taking that up as a background task but you may beat me to it.
I will work to send an updated patch by the end of the day.
Thanks,
--José
> Sincerely,
> Aaron
More information about the VDPAU
mailing list