[Mesa-dev] Status of VDPAU and XvMC state-trackers (was Re: Build error on current xvmc-r600 pipe-video)

[Jose Fonseca]

----- Original Message -----
> Am 05.06.2011 06:31, schrieb Younes Manton:
> > 2011/6/4 Jose Fonseca <jfonseca at vmware.com>:
> >> At very least there are ovious things that need to be fixed:
> >>
> >> - get_param / is_format_supported should not be duplicated from
> >> screen.
> > 
> > This is also deliberate. 

> > Params and surface formats may depend on
> > the
> > codec/profile/dimensions of the video stream the context was
> > created
> > to decode. There is not always enough info available in pipe_screen
> > alone to determine if a particular cap or surface is supported. The
> > current implementation largely wraps pipe_screen because again it's
> > using the 3D pipeline and we don't have to deal with funny decoding
> > hardware constraints.
> 
> I'm not sure if that's the right answer though, couldn't you just as
> well require a driver to handle all dimensions etc. for a given
> codec?
> If necessary should just use the shader stages (or cpu...) instead?
> 
> Also, just glancing over the interface changes:
> +enum pipe_video_codec
> +{
> +   PIPE_VIDEO_CODEC_UNKNOWN = 0,
> +   PIPE_VIDEO_CODEC_MPEG12,   /**< MPEG1, MPEG2 */
> +   PIPE_VIDEO_CODEC_MPEG4,    /**< DIVX, XVID */
> +   PIPE_VIDEO_CODEC_VC1,      /**< WMV */
> +   PIPE_VIDEO_CODEC_MPEG4_AVC /**< H.264 */
> +};
> +
> +enum pipe_video_profile
> +{
> +   PIPE_VIDEO_PROFILE_UNKNOWN,
> +   PIPE_VIDEO_PROFILE_MPEG1,
> +   PIPE_VIDEO_PROFILE_MPEG2_SIMPLE,
> +   PIPE_VIDEO_PROFILE_MPEG2_MAIN,
> +   PIPE_VIDEO_PROFILE_MPEG4_SIMPLE,
> +   PIPE_VIDEO_PROFILE_MPEG4_ADVANCED_SIMPLE,
> +   PIPE_VIDEO_PROFILE_VC1_SIMPLE,
> +   PIPE_VIDEO_PROFILE_VC1_MAIN,
> +   PIPE_VIDEO_PROFILE_VC1_ADVANCED,
> +   PIPE_VIDEO_PROFILE_MPEG4_AVC_BASELINE,
> +   PIPE_VIDEO_PROFILE_MPEG4_AVC_MAIN,
> +   PIPE_VIDEO_PROFILE_MPEG4_AVC_HIGH
> +};
> Do you really need both here?
> 
> @@ -229,9 +229,27 @@ enum pipe_format {
>     PIPE_FORMAT_L32A32_FLOAT            = 161,
>     PIPE_FORMAT_I32_FLOAT               = 162,
> 
> +   PIPE_FORMAT_YV12                    = 163,
> +   PIPE_FORMAT_YV16                    = 164,
> +   PIPE_FORMAT_IYUV                    = 165,  /**< aka I420 */
> +   PIPE_FORMAT_NV12                    = 166,
> +   PIPE_FORMAT_NV21                    = 167,
> +   PIPE_FORMAT_AYUV                    = PIPE_FORMAT_A8R8G8B8_UNORM,
> +   PIPE_FORMAT_VUYA                    = PIPE_FORMAT_B8G8R8A8_UNORM,
> +   PIPE_FORMAT_XYUV                    = PIPE_FORMAT_X8R8G8B8_UNORM,
> +   PIPE_FORMAT_VUYX                    = PIPE_FORMAT_B8G8R8X8_UNORM,
> +   PIPE_FORMAT_IA44                    = 168,
> +   PIPE_FORMAT_AI44                    = 169,
> +
>     PIPE_FORMAT_COUNT
>  };
> Defining these formats as another format feels wrong. There might be
> reasons why you'd want to use them in normal 3d contexts (ok maybe
> not
> really) but if you can't distinguish the format that's a no go.

Yes this is also incorrect. Blitting from PIPE_FORMAT_AYUV to PIPE_FORMAT_A8R8G8B8_UNORM is not a no-op.
I actually have llvmpipe AYUV support implemented in a private branch.

> Frankly I'm not sure if all these formats really should be simple
> PIPE_FORMATs even, since chances you can use them in normal 3d
> contexts
> are next to zero anyway (especially the planar stuff hurts). 

That's fine. Pixel formats just need to uniquely describe out to interpret the pixels. A 3d context doesn't need to support all of them.

I'll see
> though where that's coming from (as pipe_surface, pipe_sampler_state
> and
> friends are reused, even though the entry points are not). Though I'm
> not sure the all-new-entry points with reused gallium structs is
> really
> the right approach. Maybe if you need separate contexts etc. anyway
> (to
> be able to exploit video hardware) it would be better if you'd just
> use
> all your own structs better suited for video tasks. The vl code could
> then translate that stuff to "normal" gallium.
> If others are happy with the interface, I won't object though. I've
> no
> clue really how a better interface would look like...

My gut feel looking at [1] is that pipe_video_context interface should be either an extension of pipe_context, or optional entry-points in pipe_context. Because there's a lot of functionality needed for a pipe_context (low level resource management, relocations, fences), that will be definitely useful for video processing too.  And instead of duplicating/mirroring entry-points the existing entry-points should be enlarged to convey the necessary semantics. 

But honestly, why don't we simply move pipe_video_context out of src/gallium/include into the state tracker for the time being, and leave the admittedly complex problem of designing the gallium interface until there is an actual use case / proof-of-concept in hardware? This means we can use the video state tracker without having to make boiler plate changes for every single pipe driver, nor advertising an interface for which there's no confidence it is adequate.

Jose

[1] http://msdn.microsoft.com/en-us/library/ff563972(v=vs.85).aspx