[VDPAU] [PATCH 0/1] Add new frame and field mode chroma types. Add VdpDecoderQueryProfileCapability API

[Manoj Bonda]

Hi Philip, 

As stated in the interop2 spec (https://github.com/KhronosGroup/OpenGL-Registry/pull/223/files )

I would like to restate below points

1) Existing chroma types, existing interop API: Always fields for interop,
    driver does field splitting if necessary.

2) Existing chroma types, new interop API. Application can choose between 
    fields/frames for interop, driver does field combining / field splitting
    if necessary.

3) New chroma types, existing or new interop API. Driver never performs 
    internal field/frame conversion, allowing app to guarantee it uses the 
    most efficient path. This requires knowing the interop format before 
    performing a decode so that HW that can decode to both formats will decode
    to the format that will eventually be used for interop.
    
So clients which are fine with driver internal field combining / field splitting 
can continue to use existing chroma types with new interop API to get frames 
for interop.

new VDP_CHROMA_TYPES only provide optimized path. Applications which are fine
with re-interleaving step in driver can use the existing VDP_CHROMA_TYPES and 
the VDPAU implementation will perform this re-interleaving itself as a separate step.

Thanks,
ManojGupta.

> -----Original Message-----
> From: VDPAU <vdpau-bounces at lists.freedesktop.org> On Behalf Of Stephen
> Warren
> Sent: Tuesday, November 6, 2018 4:34 AM
> To: Philip Langdale <philipl at overt.org>; vdpau at lists.freedesktop.org
> Subject: Re: [VDPAU] [PATCH 0/1] Add new frame and field mode chroma types.
> Add VdpDecoderQueryProfileCapability API
> 
> Philip Langdale wrote at Friday, November 2, 2018 2:43 PM:
> > On Fri, 2 Nov 2018 19:59:10 +0000
> > Stephen Warren <swarren at nvidia.com> wrote:
> >
> > > I would not expect any client to do an explicit separate
> > > re-interleave step. That would certainly introduce the inefficiency
> > > we're trying to avoid. Rather, I assume that any application
> > > performing interop has some specific processing it wants to perform
> > > on the surface, and hence uses a shader to read the input surface,
> > > process the data, and then write the results to a new surface. The
> > > process of re-interleaving the input data can be completely hidden
> > > in the existing surface fetching part of the shader, without
> > > introducing any separate temporary surface or any extra copy operation.
> > >
> > > If the VDPAU implementation is forced to provide interop surfaces in
> > > a particular format that the video decoder HW itself doesn't
> > > support, then the VDPAU implementation must perform this
> > > re-interleaving itself as a separate step, rather than combining it
> > > with the desired application processing. This is what we're trying to avoid.
> >
> > Ok, so this is the disconnect. mpv most definitely doesn't try and
> > integrate the reinterleave into the other post processing it does. it
> > just has a work-around in place when vdpau is used: reinterleave as a
> > copy and then pass on to the regular post-processing pipeline. I
> > cannot speak for other media players, but I would imagine they do the
> > same thing - they have to support many different backends and trying
> > to elide this extra copy means having additional versions of various
> > post processing stages.
> >
> > So, assuming you did what I suggested, and said that interop2 was the
> > frame output version, we'd just add a conditional based on extension
> > availability and skip the reinterleave step. We wouldn't care if an
> > extra copy was being done in the driver or not - that's up to the
> > internal representation.
> >
> > > > In other words, no matter what, we've already established that the
> > > > existing interop API returns separate fields and a client must
> > > > expect that. This reality caused grief for other vendors trying to
> > > > implement interop when their hardware returned whole frames.
> > >
> > > At least the extra work is the easy way around; providing two
> > > separate sub-surfaces to interop with doubled "stride" is much
> > > easier than trying to transparently combine two separate surfaces
> > > into one for interop without being forced to just copy the surface
> > > and be
> > > inefficient:-)
> >
> > True, and this speaks to another point - if we assume that media
> > players will actively want to use interop2, then you should just
> > default to whole-frame layout in the driver whenever possible and if
> > you have to split fields for an interop1 client, it's fine in
> > practice, and if you have to reinterleave, that's work that was going
> > to happen anyway.
> 
> The approach you suggest does indeed make life simple for simple clients.
> However, it also prevents any more advanced clients from operating in a way
> that completely avoids these driver-internal surface copies in all situations. I'd
> like the API to expose enough information and/or features to allow those copies
> to be avoided where possible. That requires deciding at surface creation time, or
> at least some time before any decode operation uses the surface, how interop
> will expose those surfaces. That's what drove the current API design.
> 
> This approach doesn't prevent the later introduction of an interop v2 API that
> always exposes frame surfaces too; it's just that's not what this change is about.
> 
> --
> nvpublic
> 
> _______________________________________________
> VDPAU mailing list
> VDPAU at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/vdpau