[Spice-devel] [PATCH RFC 00/12] Remote Virgl support

[Frediano Ziglio]

> 
> Hi,
> 
> > > What is the state of the hardware supported encoding?
> > > How can we pass buffers to the hardware encoder?
> > 
> > The state here is a bit of a mess.
> > One reason to pass texture instead of dma buffers is that we use gstreamer
> > and gstreamer for hardware acceleration uses VAAPI and one way to pass
> > frames to VAAPI is to use GL textures. There is a quite strong requirement
> > that dma buffers should be mmap-able in gstreamer but this is not true.
> > Note that in theory VAAPI can import DRM prime/dma buffers however this is
> > currently not exported/implemented by gstreamer-vaapi.
> 
> Any chance to extend gstreamer-vaapi?
> 

Had a chat with Wim Taymans, I hope to have some progress on this with some
more clear understanding.

> > The current status of hardware encoding is a bit confusing.
> > On one side there is VAAPI which should be an independent (from card brand)
> > library to use hardware decoding/encoding however some vendor (like Nvidia)
> > seems not really keen on supporting it for encoding (the binding for Nvidia
> > is using vdpau which is limited to decoding). VAAPI was proposed by Intel
> > so for Intel is really good.
> 
> Hmm.  But vaapi support is pretty much required to have gstreamer handle
> video encoding for us I guess?
> 

No, current code can handle. Just using more CPU but is not really killing.

> > On the other side we could have patent/licensing issues due to the fact
> > that
> > main encoding supported (basically mpeg2, h264, hevc) all have patents
> > while
> > more open encoding (vp8, vp9) are not currently widely supported.
> 
> That is nasty indeed.  Recent intel hardware supports vp8 + vp9 too.
> Nvidia is H.264 only as far I know.
> 
> Not sure if offloading the encoding to the hardware helps with the
> patent situation.  At least we don't have to ship a (cpu) software
> encoder then.  But possibly some kind of firmware or gpu program must be
> uploaded ...
> 

I think also Nvidia did some progress with vp8/vp9.
For Nvidia you have to install NvEnc or Grid SDK which is not really
easy. The Intel software is available in fusion repository, I think
you are not completely free to install on any country, not sure about
that.

> > >  (1) Extend the display channel interface to have callbacks for these
> > >      (and thin wrapper functions which map spice display channel to
> > >      QemuConsole so spice-server doesn't need to worry about that).
> > > 
> > 
> > So you mean a way for spice-server display channel to call some Qemu
> > function, right?
> 
> Yes.  Add function pointers to QXLInterface & raise minor display
> channel interface version should do.
> 
> > >  (2) Have qemu create one context per spice-server (or per display
> > >      channel) and create a new spice_server_set_egl_context() function
> > >      to hand over the context to spice-server.
> > > 
> > 
> > Yes, I added a spice_qxl_gl_init function which set display and context.
> 
> Ah, didn't notice that on the first check.  Yes, that looks good
> interface-wise.  Possibly we should create a new context instead of
> reusing qemu_egl_rn_ctx.  But that doesn't affect the spice-server
> interface.
> 
> But can you make that a separate patch please?
> 

We are also discussing on https://lists.freedesktop.org/archives/spice-devel/2016-July/030830.html
(mail on this ML too).
Yes, possibly just passing EGLDisplay (end using DRM prime handle) will solve lot of problems.
I'll try to do some progress with VAAPI (passing DRM prime) and separating the patch.

> > Probably I feel more confident having more flexibility as is not clear
> > the resulting information we need.
> 
> I surely don't want to rush things on the interface side.  I think we
> should have at least a proof-of-concept implementation showing the
> qemu/spice-server we created actually works before merging things.
> 

Actually I can easily play games.

> > Another stuff I would like to have changed in Qemu is the number of
> > pending frames sent by it. I think that a single frame is not enough.
> > There should be at least 2/3 frames. The reason is that streaming/network
> > requires more time and would be better if there could be one frame which
> > is encoding/pending and one which is new which could be replaced by a
> > new frame that will arrive if encoding/network is not fast enough.
> 
> Hmm.  The guest will not give us 2-3 frames though.  We'll have either
> one or two, depending on whenever the guest uses page-flips or not.  So
> implementing a buffering scheme as outlined above requires us to copy
> (or let the gpu copy) the frames.
> 
> Question is where to do it best.  It is doable in qemu and spice-server.
> But spice-server knows more about the network/encoder state and can
> possibly avoid the copy, for example in case it isn't going the encode
> the frame anyway due to full network buffers, or in case no client is
> connected in the first place.
> 
> cheers,
>   Gerd
> 
> 

Probably 2 are enough, 1 I have to think about it, there may be some
possible timing problems. What happens if single frame processing
take too much? Does guest slow down?

Frediano