[VDPAU] [PATCH 0/1] Add new frame and field mode chroma types. Add VdpDecoderQueryProfileCapability API

[Stephen Warren]

Philip Langdale wrote at Friday, November 2, 2018 2:43 PM:
> On Fri, 2 Nov 2018 19:59:10 +0000
> Stephen Warren <swarren at nvidia.com> wrote:
> 
> > I would not expect any client to do an explicit separate re-interleave
> > step. That would certainly introduce the inefficiency we're trying to
> > avoid. Rather, I assume that any application performing interop has
> > some specific processing it wants to perform on the surface, and
> > hence uses a shader to read the input surface, process the data, and
> > then write the results to a new surface. The process of
> > re-interleaving the input data can be completely hidden in the
> > existing surface fetching part of the shader, without introducing any
> > separate temporary surface or any extra copy operation.
> >
> > If the VDPAU implementation is forced to provide interop surfaces in a
> > particular format that the video decoder HW itself doesn't support,
> > then the VDPAU implementation must perform this re-interleaving
> > itself as a separate step, rather than combining it with the desired
> > application processing. This is what we're trying to avoid.
> 
> Ok, so this is the disconnect. mpv most definitely doesn't try and
> integrate the reinterleave into the other post processing it does. it
> just has a work-around in place when vdpau is used: reinterleave as a
> copy and then pass on to the regular post-processing pipeline. I cannot
> speak for other media players, but I would imagine they do the same
> thing - they have to support many different backends and trying to
> elide this extra copy means having additional versions of various post
> processing stages.
> 
> So, assuming you did what I suggested, and said that interop2 was the
> frame output version, we'd just add a conditional based on extension
> availability and skip the reinterleave step. We wouldn't care if an
> extra copy was being done in the driver or not - that's up to the
> internal representation.
> 
> > > In other words, no matter what, we've
> > > already established that the existing interop API returns separate
> > > fields and a client must expect that. This reality caused grief for
> > > other vendors trying to implement interop when their hardware
> > > returned whole frames.
> >
> > At least the extra work is the easy way around; providing two separate
> > sub-surfaces to interop with doubled "stride" is much easier than
> > trying to transparently combine two separate surfaces into one for
> > interop without being forced to just copy the surface and be
> > inefficient:-)
> 
> True, and this speaks to another point - if we assume that media
> players will actively want to use interop2, then you should just
> default to whole-frame layout in the driver whenever possible and if
> you have to split fields for an interop1 client, it's fine in practice,
> and if you have to reinterleave, that's work that was going to happen
> anyway.

The approach you suggest does indeed make life simple for simple
clients. However, it also prevents any more advanced clients from
operating in a way that completely avoids these driver-internal surface
copies in all situations. I'd like the API to expose enough information
and/or features to allow those copies to be avoided where possible. That
requires deciding at surface creation time, or at least some time before
any decode operation uses the surface, how interop will expose those
surfaces. That's what drove the current API design.

This approach doesn't prevent the later introduction of an interop v2 API
that always exposes frame surfaces too; it's just that's not what this
change is about.

-- 
nvpublic