[VDPAU] [PATCH 0/1] Add new frame and field mode chroma types. Add VdpDecoderQueryProfileCapability API

[Philip Langdale]

On Fri, 2 Nov 2018 16:23:40 +0000
Stephen Warren <swarren at nvidia.com> wrote:

> This would be technically feasible, but would be inefficient.
> 
> The problem is that when the actual surface format differs from the
> format that interop mandates, the interop map/unmap APIs must perform
> a format-converting copy. This both wastes time and requires that
> additional memory be allocated to store the converted data.
> 
> Instead, if we allocate the video surfaces in a format that matches
> the format we wish to expose with interop, then we don't need any
> format- converting copy, and the system is as efficient as possible;
> interop map and unmap operations are simply barriers and don't
> require copies to be performed.
> 
> This is somewhat complicated by the fact that different hardware
> generations support different subsets of format, and this even differs
> between compressed media formats; in some cases we support decoding
> only to field-based surfaces (which drove the initial definition of
> the interop API), in some cases to only frame-based surfaces, and in
> some cases to either. Hence the need to explicitly expose this
> information to applications.
> 

This is the part I don't understand. When I look at a client actually
using the interop API (mpv in my case), it has to assume that the frame
is returned as split fields, with two textures per plane. It then does
a reinterleave as a second stage. In other words, no matter what, we've
already established that the existing interop API returns separate
fields and a client must expect that. This reality caused grief for
other vendors trying to implement interop when their hardware returned
whole frames.

So, that leads me to conclude that the ship has sailed on the
'interop1' API. It must return fields until the end of time to avoid
breaking consumers. If that means a wasteful de-interleave operation in
the driver/library for HEVC, so be it - it's the only way to hold to
the contract.

From that, I say that interop2 should simply be the
one-frame-one-texture API. And if that means the driver/library has to
do a reinterleave to satisfy the contract, it's also fine, because, in
practice, the consumer was going to reinterleave anyway(!) No consumer
is going to do anything with split fields except combine them, so that
is a constant cost, whichever part does it.

This approach seems really appealing to me as it doesn't require
touching any part of the actual vdpau API, which is currently
field/frame ignorant, and we've already established that the vdpau
readback API must return whole frames.

--phil