[VDPAU] [PATCH] Extend the public VDPAU API to support HEVC Decoding
Emil Velikov
emil.l.velikov at gmail.com
Fri Jul 25 15:02:27 PDT 2014
On 25/07/14 21:21, José Hiram Soltren wrote:
> This patch adds an API for player applications to utilize VDPAU for
> hardware-accelerated playback of H.265/HEVC streams. This patch references
> the ITU H.265/HEVC specification, available at:
> http://www.itu.int/rec/T-REC-H.265-201304-I
>
> The goals of this API are:
> - enable hardware accelerated decoding of HEVC content under VDPAU;
> - provide a reference implementation for HEVC hardware decoding that is
> vendor agnostic, and as close as possible to ideal;
> - provide enough data for HEVC hardware acceleration
> implementations from multiple vendors to be able to use the same API;
>
> Note that the API does need to be self documenting with Doxygen markup,
> which we will generate and post as an update to our public VDPAU
> documentation.
>
> I am also working on implementing VDPAU acceleration for HEVC decode in
> ffmpeg (libavcodec) and mplayer. The final version of those patches is
> dependent on a finalized API.
>
> In particular, I would appreciate feedback from anyone who might implement
> a VDPAU player backend based on this specification, either targeted at
> software decoding, or hardware accelerated playback. I would equally
> appreciate review and feedback from anyone who might add VDPAU support for
> HEVC to a player application using this API.
>
Hi José,
AFAIK most open-source vdpau backend implementation reside inside the mesa
project [1]. And at the same time I'm not sure how many mesa developers keep
an eye on this list. You might be interested in Cc-ing their mailing list [2]
One note (perhaps this is a bit premature) - but shouldn't one bump the
VDPAU{,_INFERFACE}_VERSION with such a change ?
Cheers,
Emil
[1] mesa3d.org
[2] mesa-dev at lists.freedesktop.org (no subscription is needed to post)
> Signed-off-by: José Hiram Soltren <jsoltren at nvidia.com>
> ---
> include/vdpau/vdpau.h | 294 ++++++++++++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 293 insertions(+), 1 deletion(-)
>
> diff --git a/include/vdpau/vdpau.h b/include/vdpau/vdpau.h
> index 6150607..a6f883d 100644
> --- a/include/vdpau/vdpau.h
> +++ b/include/vdpau/vdpau.h
> @@ -6,7 +6,7 @@
> /*
> * This copyright notice applies to this header file:
> *
> - * Copyright (c) 2008-2010 NVIDIA Corporation
> + * Copyright (c) 2008-2014 NVIDIA Corporation
> *
> * Permission is hereby granted, free of charge, to any person
> * obtaining a copy of this software and associated documentation
> @@ -432,6 +432,20 @@
> * Include all slices beginning with start codes 0x000001B6. The slice start
> * code must be included for all slices.
> *
> + * \subsection bitstream_hevc HEVC/H.265 - High Efficiency Video Codec
> + *
> + * Include all NAL units which encode picture data (0, 1, 2, 3, 4, 5, 6, 7, 8,
> + * 9, 16, 17, 18, 19, 20, 21). The complete slice start code (including the
> + * 0x000001 prefix) must be included for all slices, even when the prefix is
> + * not included in the bitstream.
> + *
> + * Note that if desired:
> + *
> + * - The slice start code prefix may be included in a separate bitstream
> + * buffer array entry to the actual slice data extracted from the bitstream.
> + * - Multiple bitstream buffer array entries (e.g. one per slice) may point at
> + * the same physical data storage for the slice start code prefix.
> + *
> * \section video_mixer_usage Video Mixer Usage
> *
> * \subsection video_surface_content VdpVideoSurface Content
> @@ -2437,6 +2451,15 @@ typedef uint32_t VdpDecoderProfile;
> #define VDP_DECODER_PROFILE_DIVX5_HOME_THEATER (VdpDecoderProfile)20
> /** \hideinitializer */
> #define VDP_DECODER_PROFILE_DIVX5_HD_1080P (VdpDecoderProfile)21
> +/** \hideinitializer */
> +/** \brief MPEG-H Part 2 == H.265 == HEVC */
> +#define VDP_DECODER_PROFILE_HEVC_MAIN (VdpDecoderProfile)22
> +/** \hideinitializer */
> +#define VDP_DECODER_PROFILE_HEVC_MAIN_10 (VdpDecoderProfile)23
> +/** \hideinitializer */
> +#define VDP_DECODER_PROFILE_HEVC_MAIN_12 (VdpDecoderProfile)24
> +/** \hideinitializer */
> +#define VDP_DECODER_PROFILE_HEVC_MAIN_444 (VdpDecoderProfile)25
>
> /** \hideinitializer */
> #define VDP_DECODER_LEVEL_MPEG1_NA 0
> @@ -2531,6 +2554,33 @@ typedef uint32_t VdpDecoderProfile;
> /** \hideinitializer */
> #define VDP_DECODER_LEVEL_DIVX_NA 0
>
> +/** \hideinitializer */
> +#define VDP_DECODER_LEVEL_HEVC_1 10
> +/** \hideinitializer */
> +#define VDP_DECODER_LEVEL_HEVC_2 20
> +/** \hideinitializer */
> +#define VDP_DECODER_LEVEL_HEVC_2_1 21
> +/** \hideinitializer */
> +#define VDP_DECODER_LEVEL_HEVC_3 30
> +/** \hideinitializer */
> +#define VDP_DECODER_LEVEL_HEVC_3_1 31
> +/** \hideinitializer */
> +#define VDP_DECODER_LEVEL_HEVC_4 40
> +/** \hideinitializer */
> +#define VDP_DECODER_LEVEL_HEVC_4_1 41
> +/** \hideinitializer */
> +#define VDP_DECODER_LEVEL_HEVC_5 50
> +/** \hideinitializer */
> +#define VDP_DECODER_LEVEL_HEVC_5_1 51
> +/** \hideinitializer */
> +#define VDP_DECODER_LEVEL_HEVC_5_2 52
> +/** \hideinitializer */
> +#define VDP_DECODER_LEVEL_HEVC_6 60
> +/** \hideinitializer */
> +#define VDP_DECODER_LEVEL_HEVC_6_1 61
> +/** \hideinitializer */
> +#define VDP_DECODER_LEVEL_HEVC_6_2 62
> +
> /**
> * \brief Query the implementation's VdpDecoder capabilities.
> * \param[in] device The device to query.
> @@ -2997,6 +3047,248 @@ typedef VdpPictureInfoMPEG4Part2 VdpPictureInfoDivX4;
> typedef VdpPictureInfoMPEG4Part2 VdpPictureInfoDivX5;
>
> /**
> + * \brief Picture parameter information for an HEVC/H.265 picture.
> + *
> + * Note: References to "copy of ... bitstream field" in the field
> + * descriptions may refer to data literally parsed from the bitstream,
> + * or derived from the bitstream using a mechanism described in
> + * the HEVC specification.
> + *
> + * Client applications must supply every field in this struct.
> + */
> +typedef struct {
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint32_t pic_width_in_luma_samples;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint32_t pic_height_in_luma_samples;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint8_t log2_min_luma_coding_block_size_minus3;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint8_t log2_diff_max_min_luma_coding_block_size;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint8_t log2_min_transform_block_size_minus2;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint8_t log2_diff_max_min_transform_block_size;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint8_t pcm_enabled_flag;
> + /** Copy of the Sequence Parameter Set bitstream field.
> + Only needs to be set if pcm_enabled_flag is set. */
> + uint8_t log2_min_pcm_luma_coding_block_size_minus3;
> + /** Copy of the Sequence Parameter Set bitstream field.
> + Only needs to be set if pcm_enabled_flag is set. */
> + uint8_t log2_diff_max_min_pcm_luma_coding_block_size;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint8_t bit_depth_luma_minus8;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint8_t bit_depth_chroma_minus8;
> + /** Copy of the Sequence Parameter Set bitstream field.
> + Only needs to be set if pcm_enabled_flag is set. */
> + uint8_t pcm_sample_bit_depth_luma_minus1;
> + /** Copy of the Sequence Parameter Set bitstream field.
> + Only needs to be set if pcm_enabled_flag is set. */
> + uint8_t pcm_sample_bit_depth_chroma_minus1;
> + /** Copy of the Sequence Parameter Set bitstream field.
> + Only needs to be set if pcm_enabled_flag is set. */
> + uint8_t pcm_loop_filter_disabled_flag;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint8_t strong_intra_smoothing_enabled_flag;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint8_t max_transform_hierarchy_depth_intra;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint8_t max_transform_hierarchy_depth_inter;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint8_t amp_enabled_flag;
> + /** Copy of the Sequence Parameter Set bitstream field.
> + Only valid if chroma_format_idc == 3. */
> + uint8_t separate_colour_plane_flag;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint8_t log2_max_pic_order_cnt_lsb_minus4;
> + /** Copy of the Sequence Parameter Set bitstream field.
> + Per spec, when zero, assume short_term_ref_pic_set_sps_flag
> + is also zero. */
> + uint8_t num_short_term_ref_pic_sets;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint8_t long_term_ref_pics_present_flag;
> + /** Copy of the Sequence Parameter Set bitstream field.
> + Only needed if long_term_ref_pics_present_flag is set. */
> + uint8_t num_long_term_ref_pics_sps;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint8_t sps_temporal_mvp_enabled_flag;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint8_t sample_adaptive_offset_enabled_flag;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint8_t scaling_list_enabled_flag;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint8_t chroma_format_idc;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint8_t sps_sub_layer_ordering_info_present_flag;
> + /** Copy of the Sequence Parameter Set bitstream field. */
> + uint8_t sps_max_dec_pic_buffering_minus1;
> +
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t dependent_slice_segments_enabled_flag;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t slice_segment_header_extension_present_flag;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t sign_data_hiding_enabled_flag;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t cu_qp_delta_enabled_flag;
> + /** Copy of the Picture Parameter Set bitstream field.
> + Only needed if cu_qp_delta_enabled_flag is set. */
> + uint8_t diff_cu_qp_delta_depth;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + int8_t init_qp_minus26;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + int8_t pps_cb_qp_offset;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + int8_t pps_cr_qp_offset;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t constrained_intra_pred_flag;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t weighted_pred_flag;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t weighted_bipred_flag;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t transform_skip_enabled_flag;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t transquant_bypass_enabled_flag;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t entropy_coding_sync_enabled_flag;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t log2_parallel_merge_level_minus2;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t num_extra_slice_header_bits;
> + /** Copy of the Picture Parameter Set bitstream field.
> + Only needed if tiles_enabled_flag is set. */
> + uint8_t loop_filter_across_tiles_enabled_flag;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t pps_loop_filter_across_slices_enabled_flag;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t output_flag_present_flag;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t num_ref_idx_l0_default_active_minus1;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t num_ref_idx_l1_default_active_minus1;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t lists_modification_present_flag;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t cabac_init_present_flag;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t pps_slice_chroma_qp_offsets_present_flag;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t deblocking_filter_control_present_flag;
> + /** Copy of the Picture Parameter Set bitstream field.
> + Only valid if deblocking_filter_control_present_flag is set. */
> + uint8_t deblocking_filter_override_enabled_flag;
> + /** Copy of the Picture Parameter Set bitstream field.
> + Only valid if deblocking_filter_control_present_flag is set. */
> + uint8_t pps_deblocking_filter_disabled_flag;
> + /** Copy of the Picture Parameter Set bitstream field.
> + Only valid if deblocking_filter_control_present_flag is set. */
> + int8_t pps_beta_offset_div2;
> + /** Copy of the Picture Parameter Set bitstream field.
> + Only valid if deblocking_filter_control_present_flag is set. */
> + int8_t pps_tc_offset_div2;
> + /** Copy of the Picture Parameter Set bitstream field. */
> + uint8_t tiles_enabled_flag;
> + /** Copy of the Picture Parameter Set bitstream field.
> + Only valid if tiles_enabled_flag is set. */
> + uint8_t uniform_spacing_flag;
> + /** Copy of the Picture Parameter Set bitstream field.
> + Only valid if tiles_enabled_flag is set. */
> + uint8_t num_tile_columns_minus1;
> + /** Copy of the Picture Parameter Set bitstream field.
> + Only valid if tiles_enabled_flag is set. */
> + uint8_t num_tile_rows_minus1;
> + /** Copy of the Picture Parameter Set bitstream field.
> + Only need to set 0..num_tile_columns_minus1. The struct
> + definition reserves up to the maximum of 22. */
> + uint16_t column_width_minus1[22];
> + /** Copy of the Picture Parameter Set bitstream field.
> + Only need to set 0..num_tile_rows_minus1. The struct
> + definition reserves up to the maximum of 20. */
> + uint16_t row_height_minus1[20];
> +
> + // Reference Picture Sets
> + /** Copy of specification field, CurrRpsIdx, see 7.4.7.1. */
> + uint32_t CurrRpsIdx;
> + /** Set to 1 if nal_unit_type is equal to IDR_W_RADL or IDR_N_LP.
> + Set to zero otherwise. */
> + uint32_t IDRPicFlag;
> + /** Set to 1 if nal_unit_type in the range of BLA_W_LP to
> + RSV_IRAP_VCL23, inclusive. Set to zero otherwise.*/
> + uint32_t RAPPicFlag;
> + /** Corresponds to specification field, NumDeltaPocs[RefRpsIdx].
> + Only applicable when short_term_ref_pic_set_sps_flag == 0.
> + Implementations will ignore this value in other cases. */
> + uint32_t NumDeltaPocsOfRefRpsIdx;
> + /** Copy of specification field, see Section 7.4.7.2 of the
> + HEVC specification. */
> + uint32_t NumPocTotalCurr;
> + /** Copy of specification field, see Section 8.3.2 of the
> + HEVC specification. */
> + uint32_t NumPocStCurrBefore;
> + /** Copy of specification field, see Section 8.3.2 of the
> + HEVC specification. */
> + uint32_t NumPocStCurrAfter;
> + /** Copy of specification field, see Section 8.3.2 of the
> + HEVC specification. */
> + uint32_t NumPocLtCurr;
> + /** Section 7.6.3.1 of the HEVC specification defines the syntax of
> + the slice_segment_header. This header contains information that
> + some VDPAU implementations may choose to skip. The VDPAU API
> + requires client applications to track the number of bits used in the
> + slice header for structures associated with short term and long term
> + reference pictures. First, VDPAU requires the number of bits used by
> + the short_term_ref_pic_set array in the slice_segment_header. */
> + uint32_t NumShortTermPictureSliceHeaderBits;
> + /** Second, VDPAU requires the number of bits used for long term reference
> + pictures in the slice_segment_header. This is equal to the number
> + of bits used for the contents of the block beginning with
> + "if(long_term_ref_pics_present_flag)". */
> + uint32_t NumLongTermPictureSliceHeaderBits;
> + /** The value of PicOrderCntVal of the picture in the access unit
> + containing the SEI message. The picture being decoded. */
> + int32_t CurrPicOrderCntVal;
> + /** Array of video reference surfaces.
> + Set any unused positions to VDP_INVALID_HANDLE. */
> + VdpVideoSurface RefPics[16];
> + /** Array of picture order counts. These correspond to positions
> + in the RefPics array. */
> + int32_t PicOrderCntVal[16];
> + /** Array used to specify whether a particular RefPic is
> + a long term reference. A value of "1" indicates a long-term
> + reference. */
> + uint8_t IsLongTerm[16];
> + /** Reference Picture Set list, one of the short-term RPS. */
> + int8_t RefPicSetStCurrBefore[8];
> + /** Reference Picture Set list, one of the short-term RPS. */
> + int8_t RefPicSetStCurrAfter[8];
> + /** Reference Picture Set list, one of the long-term RPS. */
> + int8_t RefPicSetLtCurr[8];
> +
> + // scaling lists (diag order)
> + /** Scaling List for 4x4 quantization matrix,
> + indexed as ScalingList4x4[matrixId][i]. */
> + uint8_t ScalingList4x4[6][16];
> + /** Scaling List for 8x8 quantization matrix,
> + indexed as ScalingList8x8[matrixId][i]. */
> + uint8_t ScalingList8x8[6][64];
> + /** Scaling List for 16x16 quantization matrix,
> + indexed as ScalingList16x16[matrixId][i]. */
> + uint8_t ScalingList16x16[6][64];
> + /** Scaling List for 32x32 quantization matrix,
> + indexed as ScalingList32x32[matrixId][i]. */
> + uint8_t ScalingList32x32[2][64];
> + /** Scaling List DC Coefficients for 16x16,
> + indexed as ScalingListDCCoeff16x16[matrixId]. */
> + uint8_t ScalingListDCCoeff16x16[6];
> + /** Scaling List DC Coefficients for 32x32,
> + indexed as ScalingListDCCoeff32x32[matrixId]. */
> + uint8_t ScalingListDCCoeff32x32[2];
> +} VdpPictureInfoHEVC;
> +
> +/**
> * \brief Decode a compressed field/frame and render the result
> * into a \ref VdpVideoSurface "VdpVideoSurface".
> * \param[in] decoder The decoder object that will perform the
>
More information about the VDPAU
mailing list