[Spice-devel] RFC: Integrating Virgil and Spice

[Marc-André Lureau]

----- Original Message -----
> On Thu, Oct 10, 2013 at 10:50 PM, Marc-André Lureau <mlureau at redhat.com>
> wrote:
> >
> >
> > ----- Original Message -----
> >> Hi,
> >>
> >> On 10/10/2013 01:25 PM, Gerd Hoffmann wrote:
> >> >    Hi,
> >> >
> >> > Nice summary.
> >> >
> >> >> 3) Virgil will render using the host gpu, using EGL to talk to
> >> >> a drm render node. For non local displays the rendered contents
> >> >> will be read back from the gpu and then passed as a pixmap to the
> >> >> ui to transport over the network
> >> >
> >> > Interesting in this context:  What is the status of 3d support for
> >> > qxl/spice?
> >>
> >> Non existent AFAIK
> >>
> >> > Is it be possible to transform virgil 3d ops into spice 3d
> >> > ops, so you could offload the rendering to spice-client?  Does it make
> >> > sense to try?  Or would the transfer of the data needed to render more
> >> > expensive than transferring the rendered screen?
> >>
> >> AFAIK, people more knowledgable people then me on 3d (ie Keith Packard)
> >> all seem to agree on that transfering the commands to render would be
> >> more expensive. IOW adding 3d support to Spice would be not really useful.
> >
> > afaik, opengl has been designed originally with remote rendering in mind.
> >
> 
> OpenGL 1.0 maybe nobody has made any accommodation to remote rendering
> in years, they haven't defined GLX protocol for new extensions in
> probably 8-10 years,
> 
> The thing is 3D rendering is high bandwidth for anything non-trivial,
> the amount of data apps move to GPUs is huge for most things.

Most opengl applications, but perhaps that's not so true for desktop apps in general (including aero etc), as you noted for gnome-shell, which is more animated than most applications that just want simple smooth transitions. I am not looking at remote gaming or 3d benchmark.

Even when a lot of data moves to GPU, I wonder how much can actually be cached (ie how much is generated on CPU).
 
> > I am no opengl expert, but it probably very much depends on the kind of
> > application (Alon reported us about Android apps remoting being fine).
> > Wouldn't glx gears be fine too? ;) I think the upcost is pretty big in
> > general, because of upload of textures and data arrays which are not very
> > well compressed in raw protocol. Probably a remote protocol, like spice,
> > could help compress those (and cache on disk!). Then result can be read
> > back in some applications, but that is not always the case (even rendering
> > to texture and composition could be done remotely). Usage of readback is
> > discouraged in general. Imho, it could be worth some experiments. But
> > current local only approach is necessary anyway, and the server rendering
> > approach could be complementary too.
> 
> There are numerous massive problems with remote rendering the android
> remoting guys don't solve because they either don't need to or they
> are too hard, but for a desktop you'd have to. I'll enumerate them for
> posterity :-) Also your GL knowledge is a bit out of date :-P
> 
> 1) readback - spice currently doesn't do remote readback it always
> does pixman rendering locally when the client reads something back,
> now GL isn't pixel perfect and if we have different rendering hw or sw
> rendering on the host and the client then we'd be giving back results
> that weren't entirely accurate. Now do we just readback from the
> client then? probably going to suck. Now things like gnome-shell use a
> technique called picking on mouse movement and clicks, every mouse
> movement can cause a readback so it know what object is under the
> mouse, can you say latency enough?

This is already in opengl 1.0 iirc (so probably with remoting in mind ;), and I don't think latency for picking matters so much in remote environment (even then, when there is a mouse click/event on client side, we could already gather some picking information perhaps, to avoid round trip), I was also imagining the double rendering that you mentionned.

> 
> 2) disconnected operation - if we are remoting to a 3D client, what do
> we do when it disconnects? fallback to local sw rendering? fallback to
> local hw rendering? GL isn't pixel perfect so the results may not be
> the same as we get on the old remote, so should we be reading back
> from the clients when they've rendered to have accurate results., GL
> also has insane number of versions and extensions, do we block apps
> from using higher levels because we might get disconnected?

> So it might be possible if you had same 3D hw in the server and guest
> to do something, where'd you double render everything, and keep some
> sort of transaction log of what the client has finished with so the
> server could throw it away, but it would be a lot of work, the android
> guys afaik don't do readback and just kill the app on disconnect.
> 
> This is of course on top of the upcost you mention, which is vast for
> most apps, not so bad for gnome-shell.
> 
> The on gpu encoding seems to be what most of the solutions in the area
> are doing, vmware, nvidia grid etc, the GPU streams the rendered
> output through the h264 encoder so the cpu has a lot less to readback
> and transfer over the network, it would be what we'd like to pursue
> but the usual patent issues around h264 means we might never get
> there. I think we are hoping that something like Daala can be used,
> and it possibly has better support for straight edges and text. But
> that is a lot more long term than getting virgil and possible some
> spice integration.
> 
> Dave.
> _______________________________________________
> Spice-devel mailing list
> Spice-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/spice-devel
>