[Spice-devel] RFC: Integrating Virgil and Spice

[Dave Airlie]

On Thu, Oct 10, 2013 at 10:50 PM, Marc-André Lureau <mlureau at redhat.com> wrote:
>
>
> ----- Original Message -----
>> Hi,
>>
>> On 10/10/2013 01:25 PM, Gerd Hoffmann wrote:
>> >    Hi,
>> >
>> > Nice summary.
>> >
>> >> 3) Virgil will render using the host gpu, using EGL to talk to
>> >> a drm render node. For non local displays the rendered contents
>> >> will be read back from the gpu and then passed as a pixmap to the
>> >> ui to transport over the network
>> >
>> > Interesting in this context:  What is the status of 3d support for
>> > qxl/spice?
>>
>> Non existent AFAIK
>>
>> > Is it be possible to transform virgil 3d ops into spice 3d
>> > ops, so you could offload the rendering to spice-client?  Does it make
>> > sense to try?  Or would the transfer of the data needed to render more
>> > expensive than transferring the rendered screen?
>>
>> AFAIK, people more knowledgable people then me on 3d (ie Keith Packard)
>> all seem to agree on that transfering the commands to render would be
>> more expensive. IOW adding 3d support to Spice would be not really useful.
>
> afaik, opengl has been designed originally with remote rendering in mind.
>

OpenGL 1.0 maybe nobody has made any accommodation to remote rendering
in years, they haven't defined GLX protocol for new extensions in
probably 8-10 years,

The thing is 3D rendering is high bandwidth for anything non-trivial,
the amount of data apps move to GPUs is huge for most things.

> I am no opengl expert, but it probably very much depends on the kind of application (Alon reported us about Android apps remoting being fine). Wouldn't glx gears be fine too? ;) I think the upcost is pretty big in general, because of upload of textures and data arrays which are not very well compressed in raw protocol. Probably a remote protocol, like spice, could help compress those (and cache on disk!). Then result can be read back in some applications, but that is not always the case (even rendering to texture and composition could be done remotely). Usage of readback is discouraged in general. Imho, it could be worth some experiments. But current local only approach is necessary anyway, and the server rendering approach could be complementary too.

There are numerous massive problems with remote rendering the android
remoting guys don't solve because they either don't need to or they
are too hard, but for a desktop you'd have to. I'll enumerate them for
posterity :-) Also your GL knowledge is a bit out of date :-P

1) readback - spice currently doesn't do remote readback it always
does pixman rendering locally when the client reads something back,
now GL isn't pixel perfect and if we have different rendering hw or sw
rendering on the host and the client then we'd be giving back results
that weren't entirely accurate. Now do we just readback from the
client then? probably going to suck. Now things like gnome-shell use a
technique called picking on mouse movement and clicks, every mouse
movement can cause a readback so it know what object is under the
mouse, can you say latency enough?

2) disconnected operation - if we are remoting to a 3D client, what do
we do when it disconnects? fallback to local sw rendering? fallback to
local hw rendering? GL isn't pixel perfect so the results may not be
the same as we get on the old remote, so should we be reading back
from the clients when they've rendered to have accurate results., GL
also has insane number of versions and extensions, do we block apps
from using higher levels because we might get disconnected?

So it might be possible if you had same 3D hw in the server and guest
to do something, where'd you double render everything, and keep some
sort of transaction log of what the client has finished with so the
server could throw it away, but it would be a lot of work, the android
guys afaik don't do readback and just kill the app on disconnect.

This is of course on top of the upcost you mention, which is vast for
most apps, not so bad for gnome-shell.

The on gpu encoding seems to be what most of the solutions in the area
are doing, vmware, nvidia grid etc, the GPU streams the rendered
output through the h264 encoder so the cpu has a lot less to readback
and transfer over the network, it would be what we'd like to pursue
but the usual patent issues around h264 means we might never get
there. I think we are hoping that something like Daala can be used,
and it possibly has better support for straight edges and text. But
that is a lot more long term than getting virgil and possible some
spice integration.

Dave.