[Spice-devel] [PATCH RFC 00/12] Remote Virgl support

[Gerd Hoffmann]

  Hi,

> > What is the state of the hardware supported encoding?
> > How can we pass buffers to the hardware encoder? 
> 
> The state here is a bit of a mess.
> One reason to pass texture instead of dma buffers is that we use gstreamer
> and gstreamer for hardware acceleration uses VAAPI and one way to pass
> frames to VAAPI is to use GL textures. There is a quite strong requirement
> that dma buffers should be mmap-able in gstreamer but this is not true.
> Note that in theory VAAPI can import DRM prime/dma buffers however this is
> currently not exported/implemented by gstreamer-vaapi.

Any chance to extend gstreamer-vaapi?

> The current status of hardware encoding is a bit confusing.
> On one side there is VAAPI which should be an independent (from card brand)
> library to use hardware decoding/encoding however some vendor (like Nvidia)
> seems not really keen on supporting it for encoding (the binding for Nvidia
> is using vdpau which is limited to decoding). VAAPI was proposed by Intel
> so for Intel is really good.

Hmm.  But vaapi support is pretty much required to have gstreamer handle
video encoding for us I guess?

> On the other side we could have patent/licensing issues due to the fact that
> main encoding supported (basically mpeg2, h264, hevc) all have patents while
> more open encoding (vp8, vp9) are not currently widely supported.

That is nasty indeed.  Recent intel hardware supports vp8 + vp9 too.
Nvidia is H.264 only as far I know.

Not sure if offloading the encoding to the hardware helps with the
patent situation.  At least we don't have to ship a (cpu) software
encoder then.  But possibly some kind of firmware or gpu program must be
uploaded ...

> >  (1) Extend the display channel interface to have callbacks for these
> >      (and thin wrapper functions which map spice display channel to
> >      QemuConsole so spice-server doesn't need to worry about that).
> > 
> 
> So you mean a way for spice-server display channel to call some Qemu
> function, right?

Yes.  Add function pointers to QXLInterface & raise minor display
channel interface version should do.

> >  (2) Have qemu create one context per spice-server (or per display
> >      channel) and create a new spice_server_set_egl_context() function
> >      to hand over the context to spice-server.
> > 
> 
> Yes, I added a spice_qxl_gl_init function which set display and context.

Ah, didn't notice that on the first check.  Yes, that looks good
interface-wise.  Possibly we should create a new context instead of
reusing qemu_egl_rn_ctx.  But that doesn't affect the spice-server
interface.

But can you make that a separate patch please?

> Probably I feel more confident having more flexibility as is not clear
> the resulting information we need.

I surely don't want to rush things on the interface side.  I think we
should have at least a proof-of-concept implementation showing the
qemu/spice-server we created actually works before merging things.

> Another stuff I would like to have changed in Qemu is the number of
> pending frames sent by it. I think that a single frame is not enough.
> There should be at least 2/3 frames. The reason is that streaming/network
> requires more time and would be better if there could be one frame which
> is encoding/pending and one which is new which could be replaced by a
> new frame that will arrive if encoding/network is not fast enough.

Hmm.  The guest will not give us 2-3 frames though.  We'll have either
one or two, depending on whenever the guest uses page-flips or not.  So
implementing a buffering scheme as outlined above requires us to copy
(or let the gpu copy) the frames.

Question is where to do it best.  It is doable in qemu and spice-server.
But spice-server knows more about the network/encoder state and can
possibly avoid the copy, for example in case it isn't going the encode
the frame anyway due to full network buffers, or in case no client is
connected in the first place.

cheers,
  Gerd