[Mesa-dev] hardware xvmc video decoding with nouveau

[Younes Manton]

2011/8/8 Christian König <deathsimple at vodafone.de>:
> Am Montag, den 08.08.2011, 15:00 +0200 schrieb Maarten Lankhorst:
>> On 08/08/2011 12:10 PM, Christian König wrote:
>> > Most modern players doesn't do it like this any more, but it still seems
>> > to cause a bunch of problems when seeking or fast forward both with
>> > mplayer and xine.
>> >
>> > So I would suggest we design the interface something like this:
>> As far as I can tell, the hardware decoder I'm using has no problem with that.
>> I just queue all commands and on putsurface I flush them. In fact, with flushing
>> on putsurface it works better than other alternatives, since it can decode future
>> and current surface simultaneously. Each command has to say which surface it's
>> being applied to. :)
> Finally somebody who understands me :D
>
> The problem I'm facing here is that switching the surface a command is
> applied to is causing either a whole bunch of overhead for me (cache
> flushes) or is not possible at all because the hardware wants an command
> buffer always filled with a whole frame.
>
> The solution I've found is that I use a separate queue for each surface
> and flush each queue each time a surface is either used as a reference
> surface or with putsurface.
>
> On the down side that means to either:
>
> A) use a hashtable where I can find the buffer for a surface (which will
> slow things down a little bit)
>
> B) or implement my own video buffer with additionally to the video data
> holds a reference to the command buffer which should be used (also ugly,
> because with vdpau you just doesn't knows whether a video buffer is used
> with hardware decoding or software decoding, but maybe that is just
> another state tracker dependent thing that needs to be hidden from the
> driver)

This is actually not an ugly idea and makes the most sense. If you
need every surface to have it's own separate decode buffer for XvMC
you do that. If you need to implement decode buffers in the decoder
for VDPAU you do that also. Again, take a closer look at the patch I
sent. The problem of what to do with VDPAU's context-free surface
creation when it comes to which decoder is being used is completely
separate from this. Any driver that wants to support both shader-based
decoding and hardware decoding at runtime should use a delayed
allocation pattern similar to what the failover driver did for
HW/softpipe mixing, that is don't create the actual buffer until the
decoder sees it and at that point create the right buffer. This means
of course that a buffer allocation failure would fail later than it
would under a regular VDPAU driver, but the VDPAU API wasn't designed
with runtime switching of decoders in mind so I'm fine with that
personally.

> C) or (and that's the way I think we should follow) enables the state
> tracker to supply an additional "hint" to the decoder which command
> buffer should be used. As long as it is not problem for you to switch
> the surface on the fly you can just ignore the "hint".
>
>> > 3. We pass the render target and the buffer to begin/render/end.
>> > This enables slice level decoder to start their decoding earlier, while
>> > it still keeps the burden of figuring out where a frame start/ends to
>> > the XvMC state tracker.
>> The nouveau hardware acceleration really doesn't need more than
>> the current patch by ymanton, and the XvMC api doesn't expose this
>> kind of information, so I don't really think it makes sense to add that.
> Yeah that's what I'm talking about, the purpose of a state tracker is to
> hide all the interface details from the driver (and not the other way
> around), and since XvMC is the only interface which doesn't have a
> distinct begin/end frame call (compared to vdpau,vaapi and DXVA), and
> that it is needed by at least some implementations -> it's the job of
> the state tracker to work around that.
>
>> > So do you have time to work on this a bit more or should I take a break
>> > from the h264 implementation and try to do it right this time? Since I
>> > think I now knew what you guys need it should be possible for me to
>> > change the interface to something everybody is happy with.
>> >
>> I'm happy with the current api for nouveau for XvMC (minus bug above).
> Are you sure? Try what's happening when you just jump back and for in a
> video with mplayer. Assuming your command buffer is somehow limited in
> size you will soon get a buffer overrun, because what mplayer does is
> the following:
>
> XvMCRenderSurface(slice 0);
> XvMCRenderSurface(slice 1);
> ....
> XvMCRenderSurface(slice n);
>
> *oh crap the use has just jumped to another key frame*
> (There is no putsurface for the first set of slices)
>
> XvMCRenderSurface(slice 0);
> XvMCRenderSurface(slice 1);
> ....
> XvMCRenderSurface(slice n);
> XvMCPutSurface(...);
>
> To solve this I first tried to increase the command buffer size, but
> that's not only a waste memory it also doesn't really solves the
> problem, because nobody limits mplayer to just call XvMCRenderSurface
> over and over again.

The hardware decoder doesn't have any problem with this. It's pushbuf
has to be flushed at regular intervals anyhow so this is no different
than any other usage scenario. A sensible way to deal with this if you
have to is to simply count how many macroblocks have been submitted
for the given surface; you'll know exactly when you have a full
frame's worth of data even without the XvMCPutSurface call and you can
flush or discard the first set of data when you see the 2nd.

>> For vdpau, I wonder if it makes more sense to have each driver have its own library,
>> for example vdpau_g3dvl.so and vdpau_nouveau.so, if/when it comes to adding nouveau
>> acceleration. If you look at the vdpau wrapper, it's using dri2 to query for the driver name
>> to use. This would allow setting the right binary to use for each driver.
> Hui? There should be already separate vdpau_*.so for each build target,
> at least the r600g driver is called libvdpau_r600.so and softpipe is
> called libvdpau_softpipe.so. If you add an vdpau-nouveau target you just
> need to set LIBBASENAME in the makefile.
>
> Regards,
> Christian.
>
>