[VDPAU] [PATCH 0/1] Add new frame and field mode chroma types. Add VdpDecoderQueryProfileCapability API

[Philip Langdale]

On Fri, 2 Nov 2018 19:59:10 +0000
Stephen Warren <swarren at nvidia.com> wrote:

> I would not expect any client to do an explicit separate re-interleave
> step. That would certainly introduce the inefficiency we're trying to
> avoid. Rather, I assume that any application performing interop has
> some specific processing it wants to perform on the surface, and
> hence uses a shader to read the input surface, process the data, and
> then write the results to a new surface. The process of
> re-interleaving the input data can be completely hidden in the
> existing surface fetching part of the shader, without introducing any
> separate temporary surface or any extra copy operation.
> 
> If the VDPAU implementation is forced to provide interop surfaces in a
> particular format that the video decoder HW itself doesn't support,
> then the VDPAU implementation must perform this re-interleaving
> itself as a separate step, rather than combining it with the desired
> application processing. This is what we're trying to avoid.

Ok, so this is the disconnect. mpv most definitely doesn't try and
integrate the reinterleave into the other post processing it does. it
just has a work-around in place when vdpau is used: reinterleave as a
copy and then pass on to the regular post-processing pipeline. I cannot
speak for other media players, but I would imagine they do the same
thing - they have to support many different backends and trying to
elide this extra copy means having additional versions of various post
processing stages.

So, assuming you did what I suggested, and said that interop2 was the
frame output version, we'd just add a conditional based on extension
availability and skip the reinterleave step. We wouldn't care if an
extra copy was being done in the driver or not - that's up to the
internal representation.
 
> > In other words, no matter what, we've
> > already established that the existing interop API returns separate
> > fields and a client must expect that. This reality caused grief for
> > other vendors trying to implement interop when their hardware
> > returned whole frames.  
> 
> At least the extra work is the easy way around; providing two separate
> sub-surfaces to interop with doubled "stride" is much easier than
> trying to transparently combine two separate surfaces into one for
> interop without being forced to just copy the surface and be
> inefficient:-)

True, and this speaks to another point - if we assume that media
players will actively want to use interop2, then you should just
default to whole-frame layout in the driver whenever possible and if
you have to split fields for an interop1 client, it's fine in practice,
and if you have to reinterleave, that's work that was going to happen
anyway.

--phil