[Spice-devel] [Qemu-devel] [PATCH Qemu] Change spice-server protocol for GL texture passing

[Frediano Ziglio]

> Hi
> 
> ----- Original Message -----
> > > 
> > > Hi
> > > 
> > > ----- Original Message -----
> > > > Forgot to add RFC to the subject
> > > > 
> > > 
> > > What's the rationale? if you share the texture id, you must share the GL
> > > context too, right? Why not use a lower level dmabuf fd that can be
> > > imported
> > > by the server gl context (which is also what the protocol require
> > > anyway)?
> > > 
> > 
> > Yes, the display and context are shared using spice_qxl_gl_init.
> > Importing again into a gl context would mean that you have to export the
> > DRM prime and import again in a separate (not shared) context.
> > It's also doable, just add 2 system call and wrapping/unwrapping.
> > Would be good to pass the EGLDisplay then so spice-server don't have to
> > initialize again possibly using another physical card.
> > 
> > We have 4 cases:
> > - client not connected;
> > - local client;
> > - remote client, software encoding;
> > - remote client, hardware encoding.
> > 
> 
> Before optimizing those syscalls and changing API etc, I would like to know
> if they are expensive (it's not my feeling)
> 
> Also, it is possible virglrenderer could be optimized to avoid exporting the
> prime fd for each scanout, if the backing image is always the same.
> 
> Sharing a GL context brings new issues. If spice server could use its own
> context, we have some context isolation (gl is still bad at MT iirc).
> 
> > Client not connected
> > Passing the texture is a no-operation, passing DRM prime require to
> > extract the handle and close every frame.
> > 
> > Local client
> > In this case there is no overhear, DRM prime is always extracted and
> > passed to the client
> > 
> > Remote client, software encoding
> > Due to different problems (DRM prime not mmap-able or data not portably
> > extractable) we'll need to import the DRM prime into a different EGL
> > context (not shared with the original one), create another texture,
> > extract data and free all texture/DRM prime.
> 
> I don't think we have strong reasons to support software encoding, video
> encoding is really expensive, and that mmap/copy is not going to be
> marginal, so even less these 2 syscalls.
> 

Using HW encoding is not easy at it seems:
- you have to have client supporting server HW encoders;
- you have to install additional software often closed source, accepting
  patents;
- you have to have right permission on the system.
What are you doing if these option are not respected? Do not allow
connections? Showing blank screen?
With a good (local) connection I can easily play using software MJPEG, why
we should avoid such configurations?

> > 
> > Remote client, hardware encoding
> > It's not clear if it's better to pass the DRM prime or the texture,
> > some API pass the texture. I got confirmation that gst_dmabuf_allocator_new
> > could try to use mmap in some cases so we should check this somehow
> > to make sure it does not.
> > 
> 
> We definitely don't want any mmap/copy to take place for hw encoding.
> 

Sure but that's hard to avoid fall backs with all different setups.

> > Taking into account that DRM prime came with "free" reference counting
> > creating the DRM prime from texture basically increase a counter which is
> > used by our implementation to make sure texture is still existing so
> > possibly passing texture instead of DRM prime just save a system call
> > in the normal case. I don't know what happens to the DRM object handle when
> > the texture is destroyed (in Qemu) with glDeleteTextures if bindings keep
> > texture "alive" or are all reset.
> > 
> > 
> > Could be that keeping qemu_spice_gl_scanout and
> > spice_qxl_gl_scanout_texture
> > as current implementation and adding a spice_qxl_gl_init/spice_qxl_gl_setup
> > passing just QXLInstance and EGLDisplay is a better solution.
> > 
> > Does is sound reasonable?
> 
> I wouldn't rush with API changes before we have a better idea how hw encoding
> can be done without mmap and wether its really worth it (I would rather see
> spice spawning a seperate gl context and process for the encoding than
> sharing it)
> 

I'm not rushing, this was the idea of RFC.
Spawing a process helps just to solve library licenses.
My list of patches for spice-server used the passed context just to create
a new context which is shared with the provided one, as I said using different
gl context and importing the DRM prime is a good option.

Passing the EGLDisplay from Qemu helps solving:
- double EGL initialization;
- multiple cards issues;
- -chroot/-runas Qemu options, where you loose access and you are not
  able to initialize EGL/VAAPI again.
I can see that Qemu searching for the card is different from VAAPI.
In case of multiple cards and Qemu run as a daemon (not having Xwayland/X)
you can end up using two physical cards.

I'll try VAAPI DRM prime passing, I hope this week.

Frediano