[VDPAU] [PATCH] Add HEVC Main 10/12 and HEVC Main 444 10/12 decode support in VDPAU API

Wed Feb 12 19:14:16 UTC 2020

On 2020-02-10 21:57, ManojGuptaBonda wrote:
> Add new profiles present in Range Extension profiles.
> 
> A.3.5 in Annexures of HEVC draft mentions of various profiles
> Main 12,
> Main 444,
> Main 444 10/12,
> Main 422 10/12,
> etc.
> under format range extension profiles. All these profiles have single
> profile_idc value - 4. These are distinguished by additional flags
> present in the bistream specified in Table A.2 of HEVC draft.
> 
> FFmpeg skips parsing the additional flags and maintains single profile
> for all the format range extension profiles. However since various GPU
> generations support various features, we are adding all the profiles
> separately so that support can be specified accurately.
> 
> This change adds the new HEVC profiles in VDPAU.h
> 
> Add new VdpPictureInfoHEVCRangeExt picture parameter structure.
> As HEVC444 profile falls under Format Range Extension profiles, due to
> similarities between Format Range Extensions profiles, the picture
> parameter structure is re-used.
> 
> 
> Also adding new VdpChromaTypes for 16bit.
> 
> For 10/12 bit HEVC decoding, the output memory layout will be P010/P012
> respectively which are similar to P016 format i.e. each component is of
> 2 Byte width (16bit) with MSBs as valid bits as per the format.
> 
> Hence adding 16bit surfaces which can accommodate both P010 and P012
> formats and can be used for both 10/12 bit decoding.
> 
> This change also adds VDP_YCBCR_FORMAT_P016, VDP_YCBCR_FORMAT_P010 
> YCrCb
> formats to be used by get/put bits functions(transfer functions) to
> access the 16 bit decoded surface.
> 
> This change adds only 16 bit decode support. 16 bit presentation is
> outside the scope of current change.
> ---
>  include/vdpau/vdpau.h | 133 ++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 133 insertions(+)
> 
> diff --git a/include/vdpau/vdpau.h b/include/vdpau/vdpau.h
> index 3533bbe..c79724d 100644
> --- a/include/vdpau/vdpau.h
> +++ b/include/vdpau/vdpau.h
> @@ -897,6 +897,65 @@ typedef uint32_t VdpChromaType;
>   *  interoped with OpenGL if the matching field/frame structure is
>   *  specified in the OpenGL API */
>  #define VDP_CHROMA_TYPE_444_FRAME ((VdpChromaType)8)
> +/** \hideinitializer \brief 4:2:0 chroma format. Undefined field/frame 
> based
> + *  Video surfaces allocated with this chroma type have undefined
> + *  field/frame structure. The implementation is free to internally 
> morph
> + *  the surface between frame/field(NV12/NV24) as required by
> + *  VdpVideoDecoder operation. Interop with OpenGL allows registration
> + *  of these surfaces for either field- or frame-based interop. But,
> an implicit
> + *  field/frame structure conversion may be performed.
> + */
> +#define VDP_CHROMA_TYPE_420_16 ((VdpChromaType)9)

Technically NV12/NV24 are 8bit specific formats and so shouldn't be used 
as
examples here. That's true for the other comments below too.

> +/** \hideinitializer \brief 4:2:2 chroma format. Undefined field/frame 
> based
> + *  Video surfaces allocated with this chroma type have undefined
> + *  field/frame structure. The implementation is free to internally 
> morph
> + *  the surface between frame/field(NV12/NV24) as required by
> + *  VdpVideoDecoder operation. Interop with OpenGL allows registration
> + *  of these surfaces for either field- or frame-based interop. But,
> an implicit
> + *  field/frame structure conversion may be performed.
> + */
> +#define VDP_CHROMA_TYPE_422_16 ((VdpChromaType)10)
> +/** \hideinitializer \brief 4:4:4 chroma format. Undefined field/frame 
> based
> + *  Video surfaces allocated with this chroma type have undefined
> + *  field/frame structure. The implementation is free to internally 
> morph
> + *  the surface between frame/field(NV12/NV24) as required by
> + *  VdpVideoDecoder operation. Interop with OpenGL allows registration
> + *  of these surfaces for either field- or frame-based interop. But,
> an implicit
> + *  field/frame structure conversion may be performed.
> + */
> +#define VDP_CHROMA_TYPE_444_16 ((VdpChromaType)11)
> +
> +/** \hideinitializer \brief 4:2:0 chroma format. Field based.
> + *  Video surfaces allocated with this chroma type can only be
> + *  interoped with OpenGL if the matching field/frame structure is
> + *  specified in the OpenGL API */
> +#define VDP_CHROMA_TYPE_420_FIELD_16 ((VdpChromaType)12)
> +/** \hideinitializer \brief 4:2:2 chroma format. Field based.
> + *  Video surfaces allocated with this chroma type can only be
> + *  interoped with OpenGL if the matching field/frame structure is
> + *  specified in the OpenGL API */
> +#define VDP_CHROMA_TYPE_422_FIELD_16 ((VdpChromaType)13)
> +/** \hideinitializer \brief 4:4:4 chroma format. Field based.
> + *  Video surfaces allocated with this chroma type can only be
> + *  interoped with OpenGL if the matching field/frame structure is
> + *  specified in the OpenGL API */
> +#define VDP_CHROMA_TYPE_444_FIELD_16 ((VdpChromaType)14)
> +
> +/** \hideinitializer \brief 4:2:0 chroma format. Frame based.
> + *  Video surfaces allocated with this chroma type can only be
> + *  interoped with OpenGL if the matching field/frame structure is
> + *  specified in the OpenGL API */
> +#define VDP_CHROMA_TYPE_420_FRAME_16 ((VdpChromaType)15)
> +/** \hideinitializer \brief 4:2:2 chroma format. Frame based.
> + *  Video surfaces allocated with this chroma type can only be
> + *  interoped with OpenGL if the matching field/frame structure is
> + *  specified in the OpenGL API */
> +#define VDP_CHROMA_TYPE_422_FRAME_16 ((VdpChromaType)16)
> +/** \hideinitializer \brief 4:4:4 chroma format. Frame based.
> + *  Video surfaces allocated with this chroma type can only be
> + *  interoped with OpenGL if the matching field/frame structure is
> + *  specified in the OpenGL API */
> +#define VDP_CHROMA_TYPE_444_FRAME_16 ((VdpChromaType)17)
> 
>  /**
>   * \brief The set of all known YCbCr surface formats.
> @@ -1009,6 +1068,66 @@ typedef uint32_t VdpYCbCrFormat;
>   * Applications should access this data via a uint8_t pointer.
>   */
>  #define VDP_YCBCR_FORMAT_Y_U_V_444     ((VdpYCbCrFormat)7)
> +/**
> + * \hideinitializer
> + * \brief The P010 surface format.
> + *
> + * This format has two planes, a Y plane and a UV plane.
> + *
> + * The Y plane is an array of two byte sized Y components.
> + * Applications should access this data via a uint16_t pointer.
> + *
> + * The UV plane is an array of interleaved two byte sized U and V
> + * components, in the order U, V, U, V. Applications should
> + * access this data via a uint8_t pointer.
> + *
> + * Note that the P010 surface format has an identical memory
> + * layout as the P016 surface format, with bits 0 through 5
> + * set to zero.
> + */
> +#define VDP_YCBCR_FORMAT_P010           ((VdpYCbCrFormat)8)
> +/**
> + * \hideinitializer
> + * \brief The P016 surface format.
> + *
> + * This format has two planes, a Y plane and a UV plane.
> + *
> + * The Y plane is an array of two byte sized Y components.
> + * Applications should access this data via a uint16_t pointer.
> + *
> + * The UV plane is an array of interleaved two byte sized U and V
> + * components, in the order U, V, U, V. Applications should
> + * access this data via a uint8_t pointer.
> + */
> +#define VDP_YCBCR_FORMAT_P016           ((VdpYCbCrFormat)9)
> + /**
> +  * \hideinitializer
> +  * \brief The "Y_U_V_444_10" YCbCr surface format.
> +  *
> +  * This format has three planes, a Y plane, a V plane, and a U
> +  * plane.
> +  *
> +  * Each of the planes is an array of two byte-sized components.
> +  *
> +  * Applications should access this data via a uint16_t pointer.
> +  *
> +  * Note that the Y_U_V_444_10 surface format has an identical memory
> +  * layout as the Y_U_V_444_16 surface format, with bits 0 through 5
> +  * set to zero.
> +  */
> + #define VDP_YCBCR_FORMAT_Y_U_V_444_10     ((VdpYCbCrFormat)10)

FWIW, I ended up never adding this as a formal format in ffmpeg. It was
considered too weird by everyone else. So we just use the 444_16 format
all the time.

> + /**
> +  * \hideinitializer
> +  * \brief The "Y_U_V_444_16" YCbCr surface format.
> +  *
> +  * This format has three planes, a Y plane, a V plane, and a U
> +  * plane.
> +  *
> +  * Each of the planes is an array of two byte-sized components.
> +  *
> +  * Applications should access this data via a uint16_t pointer.
> +  */
> + #define VDP_YCBCR_FORMAT_Y_U_V_444_16     ((VdpYCbCrFormat)11)
> 
>  /**
>   * \brief  The set of all known RGB surface formats.
> @@ -2578,6 +2697,10 @@ typedef uint32_t VdpDecoderProfile;
>  #define VDP_DECODER_PROFILE_HEVC_MAIN_12
> ((VdpDecoderProfile)103)
>  /** \hideinitializer */
>  #define VDP_DECODER_PROFILE_HEVC_MAIN_444
> ((VdpDecoderProfile)104)
> +/** \hideinitializer */
> +#define VDP_DECODER_PROFILE_HEVC_MAIN_444_10
> ((VdpDecoderProfile)105)
> +/** \hideinitializer */
> +#define VDP_DECODER_PROFILE_HEVC_MAIN_444_12
> ((VdpDecoderProfile)106)
> 
>  /** \hideinitializer */
>  #define VDP_DECODER_LEVEL_MPEG1_NA 0
> @@ -3545,6 +3668,16 @@ typedef struct {
> 
>  } VdpPictureInfoHEVC444;
> 
> +/**
> + * \brief Picture parameter information for HEVC Range Extensions 
> picture.
> + *
> + * HEVC Main 444 Profile is part of Range Extensions profiles,
> + * Due to similarities between Range Extensions profiles, the picture
> + * parameter structure is re-used for Format Range Extensions profiles
> + * supported.
> + */
> +typedef VdpPictureInfoHEVC444 VdpPictureInfoHEVCRangeExt;
> +
>  /**
>   * \brief Decode a compressed field/frame and render the result
>   *        into a \ref VdpVideoSurface "VdpVideoSurface".

Looks fine otherwise.

--phil