[VDPAU] [PATCH 0/1] Add new frame and field mode chroma types. Add VdpDecoderQueryProfileCapability API

[Manoj Bonda]

Inline.

> -----Original Message-----
> From: VDPAU <vdpau-bounces at lists.freedesktop.org> On Behalf Of Philip
> Langdale
> Sent: Thursday, November 1, 2018 9:50 AM
> To: Manoj Bonda <mbonda at nvidia.com>
> Cc: vdpau at lists.freedesktop.org
> Subject: Re: [VDPAU] [PATCH 0/1] Add new frame and field mode chroma
> types. Add VdpDecoderQueryProfileCapability API
> 
> On Wed, 17 Oct 2018 17:46:25 +0530
> Manoj Bonda <mbonda at nvidia.com> wrote:
> 
> > The new API proposed here addresses the request to provide a new mode,
> > where the VDPAU OpenGL interop surfaces are frame or field based.
> > (Reference : https://patchwork.ffmpeg.org/patch/4172/ )
> >
> > Existing VDPAU OpenGL interop supports field-based surfaces only.
> > Applications are not aware of the picture structure of the video
> > surface nor can they allocate the video surface with a specific
> > picture structure.
> >
> > The picture structure can be either "field" or "frame". In a video
> > surface with field picture structure, odd lines are part of the top
> > field and even lines are part of the bottom field. In the case of a
> > video surface with frame structure, there are no top/bottom fields;
> > all lines are stored sequentially as a progressive frame.
> >
> > The new API allows applications to allocate surfaces with the required
> > picture structure. Chroma types defined in the VDPAU API are extended
> > for this purpose.
> >
> > For the below chroma types that already exist,
> >     VDP_CHROMA_TYPE_420
> >     VDP_CHROMA_TYPE_422
> >     VDP_CHROMA_TYPE_444
> >
> > The VDPAU implementation can either directly decode into the video
> > surface passed or decode to another internal surface with a different
> > picture structure type it supports. When used with VDPAU OpenGL
> > interop, the decoded data may be copied from a driver internal surface
> > to the application allocated surface.
> >
> > With the existing interop API, the surface is always treated as having
> > field picture structure. With the updated interop API, surfaces can be
> > registered for either field or frame surface type and used
> > accordingly.
> >
> > Applications can allocate video surfaces with this enum and use those
> > video surfaces with the interop API. This usage allows more
> > flexibility for applications and VDPAU implementations to support
> > various use cases. E.g: The application expects a frame but vdpau
> > implementation can provide field.
> >
> > This change proposes new enums:
> >     VDP_CHROMA_TYPE_420_FIELD
> >     VDP_CHROMA_TYPE_422_FIELD
> >     VDP_CHROMA_TYPE_444_FIELD
> >     VDP_CHROMA_TYPE_420_FRAME
> >     VDP_CHROMA_TYPE_422_FRAME
> >     VDP_CHROMA_TYPE_444_FRAME
> 
> My big question is why extend the Chroma Type enum in this way? Why not
> add a separate enum that specifies whether you want Field or Frame layout.
> Fudging it into the chroma type feels like a hack to avoid having to extend
> various API calls.
Separate enum will require either change in the existing VdpVideoSurfaceCreate or add 
a new surface allocation API. To continue support existing applications, we cannot change the VdpVideoSurfaceCreate.
Extending the enum along with existing API is  less invasive change than adding a new API for each
different type of surface creation.
> 
> From my perspective, I would have imagined that you would not really need
> this at all, because the field/frame layout concept is already hidden as an
> implementation detail everywhere in VDPAU except with respect to the GL
> interop. So why is it not sufficient to have
> GL_NV_vdpau_interop2 as already described, where the picture structure can
> be specified? The driver can presumably react accordingly at that time. And to
The Hardware decoder may be capable of generating output in field or frame or both of them.
The intention is to design an API that can work without assuming any specific hardware capabilities.
With above approach we have to convert surfaces from the layout in which hardware generated an output
to the layout in which application expects output. Even though that path is still supported, new optimized 
flow is also provided.
The new API that queries capability returns the underlying  hardware/implementation support, 
which is either decode to field or decode to frame or decode to either of them.
Application can choose the surface type based on hardware capability and its needs. 

> be honest, I think having the picture structure flag on the new method is
> probably unnecessary. Just have the new API call for whole frames and use the
> old API if you somehow want fields, and no one actually wants fields. The
> driver would only expose the new extension if it could actually return pictures
> as frames.
The old API doesn’t explicitly specify the output layout to be field. 
The new API explicitly calls out the layout to be field or frame.
> 
> This seems to address the problem as directly as possible.
> 
> Thanks,
> 
> --phil
Thanks,
ManojGupta.
> _______________________________________________
> VDPAU mailing list
> VDPAU at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/vdpau