[VDPAU] [PATCH] Extend the public VDPAU API to support HEVC Decoding
Christian König
deathsimple at vodafone.de
Sat Jul 26 02:42:54 PDT 2014
Am 26.07.2014 um 00:02 schrieb Emil Velikov:
> On 25/07/14 21:21, José Hiram Soltren wrote:
>> This patch adds an API for player applications to utilize VDPAU for
>> hardware-accelerated playback of H.265/HEVC streams. This patch references
>> the ITU H.265/HEVC specification, available at:
>> http://www.itu.int/rec/T-REC-H.265-201304-I
>>
>> The goals of this API are:
>> - enable hardware accelerated decoding of HEVC content under VDPAU;
>> - provide a reference implementation for HEVC hardware decoding that is
>> vendor agnostic, and as close as possible to ideal;
>> - provide enough data for HEVC hardware acceleration
>> implementations from multiple vendors to be able to use the same API;
>>
>> Note that the API does need to be self documenting with Doxygen markup,
>> which we will generate and post as an update to our public VDPAU
>> documentation.
>>
>> I am also working on implementing VDPAU acceleration for HEVC decode in
>> ffmpeg (libavcodec) and mplayer. The final version of those patches is
>> dependent on a finalized API.
>>
>> In particular, I would appreciate feedback from anyone who might implement
>> a VDPAU player backend based on this specification, either targeted at
>> software decoding, or hardware accelerated playback. I would equally
>> appreciate review and feedback from anyone who might add VDPAU support for
>> HEVC to a player application using this API.
>>
> Hi José,
> AFAIK most open-source vdpau backend implementation reside inside the mesa
> project [1]. And at the same time I'm not sure how many mesa developers keep
> an eye on this list. You might be interested in Cc-ing their mailing list [2]
I do watch this list as well, but Cc-ing mesa on such a change also
sounds like a good idea too.
> One note (perhaps this is a bit premature) - but shouldn't one bump the
> VDPAU{,_INFERFACE}_VERSION with such a change ?
Yeah, agree. But this is probably just a first try of the patch and as
it is usually with such changes whoever commits it upstream should bump
the version.
Apart from that I'm not so deep into HEVC yet so I can't judge if this
is correct or not, but going to ask around at AMD if somebody could take
a look as well.
Christian.
>
> Cheers,
> Emil
>
> [1] mesa3d.org
> [2] mesa-dev at lists.freedesktop.org (no subscription is needed to post)
>
>> Signed-off-by: José Hiram Soltren <jsoltren at nvidia.com>
>> ---
>> include/vdpau/vdpau.h | 294 ++++++++++++++++++++++++++++++++++++++++++++++++-
>> 1 file changed, 293 insertions(+), 1 deletion(-)
>>
>> diff --git a/include/vdpau/vdpau.h b/include/vdpau/vdpau.h
>> index 6150607..a6f883d 100644
>> --- a/include/vdpau/vdpau.h
>> +++ b/include/vdpau/vdpau.h
>> @@ -6,7 +6,7 @@
>> /*
>> * This copyright notice applies to this header file:
>> *
>> - * Copyright (c) 2008-2010 NVIDIA Corporation
>> + * Copyright (c) 2008-2014 NVIDIA Corporation
>> *
>> * Permission is hereby granted, free of charge, to any person
>> * obtaining a copy of this software and associated documentation
>> @@ -432,6 +432,20 @@
>> * Include all slices beginning with start codes 0x000001B6. The slice start
>> * code must be included for all slices.
>> *
>> + * \subsection bitstream_hevc HEVC/H.265 - High Efficiency Video Codec
>> + *
>> + * Include all NAL units which encode picture data (0, 1, 2, 3, 4, 5, 6, 7, 8,
>> + * 9, 16, 17, 18, 19, 20, 21). The complete slice start code (including the
>> + * 0x000001 prefix) must be included for all slices, even when the prefix is
>> + * not included in the bitstream.
>> + *
>> + * Note that if desired:
>> + *
>> + * - The slice start code prefix may be included in a separate bitstream
>> + * buffer array entry to the actual slice data extracted from the bitstream.
>> + * - Multiple bitstream buffer array entries (e.g. one per slice) may point at
>> + * the same physical data storage for the slice start code prefix.
>> + *
>> * \section video_mixer_usage Video Mixer Usage
>> *
>> * \subsection video_surface_content VdpVideoSurface Content
>> @@ -2437,6 +2451,15 @@ typedef uint32_t VdpDecoderProfile;
>> #define VDP_DECODER_PROFILE_DIVX5_HOME_THEATER (VdpDecoderProfile)20
>> /** \hideinitializer */
>> #define VDP_DECODER_PROFILE_DIVX5_HD_1080P (VdpDecoderProfile)21
>> +/** \hideinitializer */
>> +/** \brief MPEG-H Part 2 == H.265 == HEVC */
>> +#define VDP_DECODER_PROFILE_HEVC_MAIN (VdpDecoderProfile)22
>> +/** \hideinitializer */
>> +#define VDP_DECODER_PROFILE_HEVC_MAIN_10 (VdpDecoderProfile)23
>> +/** \hideinitializer */
>> +#define VDP_DECODER_PROFILE_HEVC_MAIN_12 (VdpDecoderProfile)24
>> +/** \hideinitializer */
>> +#define VDP_DECODER_PROFILE_HEVC_MAIN_444 (VdpDecoderProfile)25
>>
>> /** \hideinitializer */
>> #define VDP_DECODER_LEVEL_MPEG1_NA 0
>> @@ -2531,6 +2554,33 @@ typedef uint32_t VdpDecoderProfile;
>> /** \hideinitializer */
>> #define VDP_DECODER_LEVEL_DIVX_NA 0
>>
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_1 10
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_2 20
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_2_1 21
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_3 30
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_3_1 31
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_4 40
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_4_1 41
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_5 50
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_5_1 51
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_5_2 52
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_6 60
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_6_1 61
>> +/** \hideinitializer */
>> +#define VDP_DECODER_LEVEL_HEVC_6_2 62
>> +
>> /**
>> * \brief Query the implementation's VdpDecoder capabilities.
>> * \param[in] device The device to query.
>> @@ -2997,6 +3047,248 @@ typedef VdpPictureInfoMPEG4Part2 VdpPictureInfoDivX4;
>> typedef VdpPictureInfoMPEG4Part2 VdpPictureInfoDivX5;
>>
>> /**
>> + * \brief Picture parameter information for an HEVC/H.265 picture.
>> + *
>> + * Note: References to "copy of ... bitstream field" in the field
>> + * descriptions may refer to data literally parsed from the bitstream,
>> + * or derived from the bitstream using a mechanism described in
>> + * the HEVC specification.
>> + *
>> + * Client applications must supply every field in this struct.
>> + */
>> +typedef struct {
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint32_t pic_width_in_luma_samples;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint32_t pic_height_in_luma_samples;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t log2_min_luma_coding_block_size_minus3;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t log2_diff_max_min_luma_coding_block_size;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t log2_min_transform_block_size_minus2;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t log2_diff_max_min_transform_block_size;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t pcm_enabled_flag;
>> + /** Copy of the Sequence Parameter Set bitstream field.
>> + Only needs to be set if pcm_enabled_flag is set. */
>> + uint8_t log2_min_pcm_luma_coding_block_size_minus3;
>> + /** Copy of the Sequence Parameter Set bitstream field.
>> + Only needs to be set if pcm_enabled_flag is set. */
>> + uint8_t log2_diff_max_min_pcm_luma_coding_block_size;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t bit_depth_luma_minus8;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t bit_depth_chroma_minus8;
>> + /** Copy of the Sequence Parameter Set bitstream field.
>> + Only needs to be set if pcm_enabled_flag is set. */
>> + uint8_t pcm_sample_bit_depth_luma_minus1;
>> + /** Copy of the Sequence Parameter Set bitstream field.
>> + Only needs to be set if pcm_enabled_flag is set. */
>> + uint8_t pcm_sample_bit_depth_chroma_minus1;
>> + /** Copy of the Sequence Parameter Set bitstream field.
>> + Only needs to be set if pcm_enabled_flag is set. */
>> + uint8_t pcm_loop_filter_disabled_flag;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t strong_intra_smoothing_enabled_flag;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t max_transform_hierarchy_depth_intra;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t max_transform_hierarchy_depth_inter;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t amp_enabled_flag;
>> + /** Copy of the Sequence Parameter Set bitstream field.
>> + Only valid if chroma_format_idc == 3. */
>> + uint8_t separate_colour_plane_flag;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t log2_max_pic_order_cnt_lsb_minus4;
>> + /** Copy of the Sequence Parameter Set bitstream field.
>> + Per spec, when zero, assume short_term_ref_pic_set_sps_flag
>> + is also zero. */
>> + uint8_t num_short_term_ref_pic_sets;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t long_term_ref_pics_present_flag;
>> + /** Copy of the Sequence Parameter Set bitstream field.
>> + Only needed if long_term_ref_pics_present_flag is set. */
>> + uint8_t num_long_term_ref_pics_sps;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t sps_temporal_mvp_enabled_flag;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t sample_adaptive_offset_enabled_flag;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t scaling_list_enabled_flag;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t chroma_format_idc;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t sps_sub_layer_ordering_info_present_flag;
>> + /** Copy of the Sequence Parameter Set bitstream field. */
>> + uint8_t sps_max_dec_pic_buffering_minus1;
>> +
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t dependent_slice_segments_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t slice_segment_header_extension_present_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t sign_data_hiding_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t cu_qp_delta_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only needed if cu_qp_delta_enabled_flag is set. */
>> + uint8_t diff_cu_qp_delta_depth;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + int8_t init_qp_minus26;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + int8_t pps_cb_qp_offset;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + int8_t pps_cr_qp_offset;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t constrained_intra_pred_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t weighted_pred_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t weighted_bipred_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t transform_skip_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t transquant_bypass_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t entropy_coding_sync_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t log2_parallel_merge_level_minus2;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t num_extra_slice_header_bits;
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only needed if tiles_enabled_flag is set. */
>> + uint8_t loop_filter_across_tiles_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t pps_loop_filter_across_slices_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t output_flag_present_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t num_ref_idx_l0_default_active_minus1;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t num_ref_idx_l1_default_active_minus1;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t lists_modification_present_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t cabac_init_present_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t pps_slice_chroma_qp_offsets_present_flag;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t deblocking_filter_control_present_flag;
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only valid if deblocking_filter_control_present_flag is set. */
>> + uint8_t deblocking_filter_override_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only valid if deblocking_filter_control_present_flag is set. */
>> + uint8_t pps_deblocking_filter_disabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only valid if deblocking_filter_control_present_flag is set. */
>> + int8_t pps_beta_offset_div2;
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only valid if deblocking_filter_control_present_flag is set. */
>> + int8_t pps_tc_offset_div2;
>> + /** Copy of the Picture Parameter Set bitstream field. */
>> + uint8_t tiles_enabled_flag;
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only valid if tiles_enabled_flag is set. */
>> + uint8_t uniform_spacing_flag;
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only valid if tiles_enabled_flag is set. */
>> + uint8_t num_tile_columns_minus1;
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only valid if tiles_enabled_flag is set. */
>> + uint8_t num_tile_rows_minus1;
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only need to set 0..num_tile_columns_minus1. The struct
>> + definition reserves up to the maximum of 22. */
>> + uint16_t column_width_minus1[22];
>> + /** Copy of the Picture Parameter Set bitstream field.
>> + Only need to set 0..num_tile_rows_minus1. The struct
>> + definition reserves up to the maximum of 20. */
>> + uint16_t row_height_minus1[20];
>> +
>> + // Reference Picture Sets
>> + /** Copy of specification field, CurrRpsIdx, see 7.4.7.1. */
>> + uint32_t CurrRpsIdx;
>> + /** Set to 1 if nal_unit_type is equal to IDR_W_RADL or IDR_N_LP.
>> + Set to zero otherwise. */
>> + uint32_t IDRPicFlag;
>> + /** Set to 1 if nal_unit_type in the range of BLA_W_LP to
>> + RSV_IRAP_VCL23, inclusive. Set to zero otherwise.*/
>> + uint32_t RAPPicFlag;
>> + /** Corresponds to specification field, NumDeltaPocs[RefRpsIdx].
>> + Only applicable when short_term_ref_pic_set_sps_flag == 0.
>> + Implementations will ignore this value in other cases. */
>> + uint32_t NumDeltaPocsOfRefRpsIdx;
>> + /** Copy of specification field, see Section 7.4.7.2 of the
>> + HEVC specification. */
>> + uint32_t NumPocTotalCurr;
>> + /** Copy of specification field, see Section 8.3.2 of the
>> + HEVC specification. */
>> + uint32_t NumPocStCurrBefore;
>> + /** Copy of specification field, see Section 8.3.2 of the
>> + HEVC specification. */
>> + uint32_t NumPocStCurrAfter;
>> + /** Copy of specification field, see Section 8.3.2 of the
>> + HEVC specification. */
>> + uint32_t NumPocLtCurr;
>> + /** Section 7.6.3.1 of the HEVC specification defines the syntax of
>> + the slice_segment_header. This header contains information that
>> + some VDPAU implementations may choose to skip. The VDPAU API
>> + requires client applications to track the number of bits used in the
>> + slice header for structures associated with short term and long term
>> + reference pictures. First, VDPAU requires the number of bits used by
>> + the short_term_ref_pic_set array in the slice_segment_header. */
>> + uint32_t NumShortTermPictureSliceHeaderBits;
>> + /** Second, VDPAU requires the number of bits used for long term reference
>> + pictures in the slice_segment_header. This is equal to the number
>> + of bits used for the contents of the block beginning with
>> + "if(long_term_ref_pics_present_flag)". */
>> + uint32_t NumLongTermPictureSliceHeaderBits;
>> + /** The value of PicOrderCntVal of the picture in the access unit
>> + containing the SEI message. The picture being decoded. */
>> + int32_t CurrPicOrderCntVal;
>> + /** Array of video reference surfaces.
>> + Set any unused positions to VDP_INVALID_HANDLE. */
>> + VdpVideoSurface RefPics[16];
>> + /** Array of picture order counts. These correspond to positions
>> + in the RefPics array. */
>> + int32_t PicOrderCntVal[16];
>> + /** Array used to specify whether a particular RefPic is
>> + a long term reference. A value of "1" indicates a long-term
>> + reference. */
>> + uint8_t IsLongTerm[16];
>> + /** Reference Picture Set list, one of the short-term RPS. */
>> + int8_t RefPicSetStCurrBefore[8];
>> + /** Reference Picture Set list, one of the short-term RPS. */
>> + int8_t RefPicSetStCurrAfter[8];
>> + /** Reference Picture Set list, one of the long-term RPS. */
>> + int8_t RefPicSetLtCurr[8];
>> +
>> + // scaling lists (diag order)
>> + /** Scaling List for 4x4 quantization matrix,
>> + indexed as ScalingList4x4[matrixId][i]. */
>> + uint8_t ScalingList4x4[6][16];
>> + /** Scaling List for 8x8 quantization matrix,
>> + indexed as ScalingList8x8[matrixId][i]. */
>> + uint8_t ScalingList8x8[6][64];
>> + /** Scaling List for 16x16 quantization matrix,
>> + indexed as ScalingList16x16[matrixId][i]. */
>> + uint8_t ScalingList16x16[6][64];
>> + /** Scaling List for 32x32 quantization matrix,
>> + indexed as ScalingList32x32[matrixId][i]. */
>> + uint8_t ScalingList32x32[2][64];
>> + /** Scaling List DC Coefficients for 16x16,
>> + indexed as ScalingListDCCoeff16x16[matrixId]. */
>> + uint8_t ScalingListDCCoeff16x16[6];
>> + /** Scaling List DC Coefficients for 32x32,
>> + indexed as ScalingListDCCoeff32x32[matrixId]. */
>> + uint8_t ScalingListDCCoeff32x32[2];
>> +} VdpPictureInfoHEVC;
>> +
>> +/**
>> * \brief Decode a compressed field/frame and render the result
>> * into a \ref VdpVideoSurface "VdpVideoSurface".
>> * \param[in] decoder The decoder object that will perform the
>>
> _______________________________________________
> VDPAU mailing list
> VDPAU at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/vdpau
More information about the VDPAU
mailing list