[VDPAU] [PATCH 0/1] Add new frame and field mode chroma types. Add VdpDecoderQueryProfileCapability API

[Philip Langdale]

On Wed, 17 Oct 2018 17:46:25 +0530
Manoj Bonda <mbonda at nvidia.com> wrote:

> The new API proposed here addresses the request to provide a new mode,
> where the VDPAU OpenGL interop surfaces are frame or field based.
> (Reference : https://patchwork.ffmpeg.org/patch/4172/ )
> 
> Existing VDPAU OpenGL interop supports field-based surfaces only.
> Applications are not aware of the picture structure of the video
> surface nor can they allocate the video surface with a specific
> picture structure.
> 
> The picture structure can be either "field" or "frame". In a video
> surface with field picture structure, odd lines are part of the top
> field and even lines are part of the bottom field. In the case of a
> video surface with frame structure, there are no top/bottom fields;
> all lines are stored sequentially as a progressive frame.
> 
> The new API allows applications to allocate surfaces with the
> required picture structure. Chroma types defined in the VDPAU API are
> extended for this purpose.
> 
> For the below chroma types that already exist,
>     VDP_CHROMA_TYPE_420
>     VDP_CHROMA_TYPE_422
>     VDP_CHROMA_TYPE_444 
> 
> The VDPAU implementation can either directly decode into the video
> surface passed or decode to another internal surface with a different
> picture structure type it supports. When used with VDPAU OpenGL
> interop, the decoded data may be copied from a driver internal
> surface to the application allocated surface. 
> 
> With the existing interop API, the surface is always treated as
> having field picture structure. With the updated interop API,
> surfaces can be registered for either field or frame surface type and
> used accordingly.
> 
> Applications can allocate video surfaces with this enum and use those 
> video surfaces with the interop API. This usage allows more
> flexibility for applications and VDPAU implementations to support
> various use cases. E.g: The application expects a frame but vdpau
> implementation can provide field.
> 
> This change proposes new enums:
>     VDP_CHROMA_TYPE_420_FIELD
>     VDP_CHROMA_TYPE_422_FIELD
>     VDP_CHROMA_TYPE_444_FIELD
>     VDP_CHROMA_TYPE_420_FRAME
>     VDP_CHROMA_TYPE_422_FRAME
>     VDP_CHROMA_TYPE_444_FRAME

My big question is why extend the Chroma Type enum in this way? Why not
add a separate enum that specifies whether you want Field or Frame
layout. Fudging it into the chroma type feels like a hack to avoid
having to extend various API calls.

From my perspective, I would have imagined that you would not really
need this at all, because the field/frame layout concept is already
hidden as an implementation detail everywhere in VDPAU except with
respect to the GL interop. So why is it not sufficient to have
GL_NV_vdpau_interop2 as already described, where the picture structure
can be specified? The driver can presumably react accordingly at that
time. And to be honest, I think having the picture structure flag on
the new method is probably unnecessary. Just have the new API call
for whole frames and use the old API if you somehow want fields, and
no one actually wants fields. The driver would only expose the new
extension if it could actually return pictures as frames.

This seems to address the problem as directly as possible.

Thanks,

--phil