<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html> <body> <div dir="auto"> <div dir="auto"><span style="font-size: 12pt;">On May 26, 2021 13:15:08 Daniel Stone <daniel@fooishbar.org> wrote:</span></div><div id="aqm-original" style="color: black;"> <div><br></div> <blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #808080; padding-left: 0.75ex;"> <div dir="auto">Hey,</div> <div dir="auto"><br></div> <div dir="auto">On Wed, 26 May 2021 at 16:24, Jason Ekstrand <jason@jlekstrand.net> wrote:</div> <blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #0099CC; padding-left: 0.75ex;"> <div dir="auto">On Wed, May 26, 2021 at 6:09 AM Daniel Stone <daniel@fooishbar.org> wrote:</div> <blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #9933CC; padding-left: 0.75ex;"> <div dir="auto">Typing out the Wayland protocol isn't the hard bit. If we just need to</div> <div dir="auto">copy and sed syncobj to weirdsyncobj, no problem really, and it gives</div> <div dir="auto">us a six-month head start on painful compositor-internal surgery</div> <div dir="auto">whilst we work on common infrastructure to ship userspace fences</div> <div dir="auto">around (mappable dmabuf with the sync bracketing? FD where every</div> <div dir="auto">read() gives you the current value? memfd? other?).</div> </blockquote> <div dir="auto"><br></div> <div dir="auto">I feel like I should elaborate more about timelines. In my earlier</div> <div dir="auto">reply, my commentary about timeline syncobj was mostly focused around</div> <div dir="auto">helping people avoid typing. That's not really the full story,</div> <div dir="auto">though, and I hope more context will help.</div> <div dir="auto"><br></div> <div dir="auto">First, let me say that timeline syncobj was designed as a mechanism to</div> <div dir="auto">implement VK_KHR_timeline_semaphore without inserting future fences</div> <div dir="auto">into the kernel. It's entirely designed around the needs of Vulkan</div> <div dir="auto">drivers, not really as a window-system primitive. The semantics are</div> <div dir="auto">designed around one driver communicating to another that new fences</div> <div dir="auto">have been added and it's safe to kick off more rendering. I'm not</div> <div dir="auto">convinced that it's the right object for window-systems and I'm also</div> <div dir="auto">not convinced that it's a good idea to try and make a version of it</div> <div dir="auto">that's a wrapper around a userspace memory fence. (I'm going to start</div> <div dir="auto">typing UMF for userspace memory fence because it's long to type out.)</div> <div dir="auto"><br></div> <div dir="auto">Why? Well, the fundamental problem with timelines in general is</div> <div dir="auto">trying to figure out when it's about to be done. But timeline syncobj</div> <div dir="auto">solves this for us! It gives us this fancy super-useful ioctl!</div> <div dir="auto">Right? Uh.... not as well as I'd like. Let's say we make a timeline</div> <div dir="auto">syncobj that's a wrapper around a userspace memory fence. What do we</div> <div dir="auto">do with that ioctl? As I mentioned above, the kernel doesn't have any</div> <div dir="auto">clue when it will be triggered so that ioctl turns into an actual</div> <div dir="auto">wait. That's no good because it creates unnecessary stalls.</div> </blockquote> <div dir="auto"><br></div> <div dir="auto">Yeah, I'm assuming that UMF will be a separate primitive. No problem.</div> <div dir="auto">I also think that your submitted/completed thing is a non-problem: at</div> <div dir="auto">this stage we're just throwing up our hands and admitting that we're</div> <div dir="auto">letting userspace tie itself in knots, and giving it the tools to tie</div> <div dir="auto">a sufficiently un-streetwise compositor in knots too. We're already</div> <div dir="auto">crossing that Rubicon, so let's just embrace it and not try to design</div> <div dir="auto">it out. Us compositors can handle the scheduling, really.</div></blockquote></div><div dir="auto"><br></div><div dir="auto">Ok, good. I think we're on the same page.</div><div dir="auto"><br></div><div id="aqm-original" style="color: black;" dir="auto"><blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #808080; padding-left: 0.75ex;"><div dir="auto"></div> <blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #0099CC; padding-left: 0.75ex;"> <div dir="auto">There's another potential solution here: Have each UMF be two</div> <div dir="auto">timelines: submitted and completed. At the start of every batch</div> <div dir="auto">that's supposed to trigger a UMF, we set the "submitted" side and</div> <div dir="auto">then, when it completes, we set the "completed" side. Ok, great, now</div> <div dir="auto">we can get at the "about to be done" with the submitted side,</div> <div dir="auto">implement the ioctl, and we're all good, right? Sadly, no. There's</div> <div dir="auto">no guarantee about how long a "batch" takes. So there's no universal</div> <div dir="auto">timeout the kernel can apply. Also, if it does time out, the kernel</div> <div dir="auto">doesn't know who to blame for the timeout and how to prevent itself</div> <div dir="auto">from getting in trouble again. The compositor does so, in theory,</div> <div dir="auto">given the right ioctls, it could detect the -ETIME and kill that</div> <div dir="auto">client. Not a great solution.</div> <div dir="auto"><br></div> <div dir="auto">The best option I've been able to come up with for this is some sort</div> <div dir="auto">of client-provided signal. Something where it says, as part of submit</div> <div dir="auto">or somewhere else, "I promise I'll be done soon" where that promise</div> <div dir="auto">comes with dire consequences if it's not. At that point, we can turn</div> <div dir="auto">the UMF and a particular wait value into a one-shot fence like a</div> <div dir="auto">dma_fence or sync_file, or signal a syncobj on it. If it ever times</div> <div dir="auto">out, we kick their context. In Vulkan terminology, they get</div> <div dir="auto">VK_ERROR_DEVICE_LOST. There are two important bits here: First, is</div> <div dir="auto">that it's based on a client-provided thing. With a fully timeline</div> <div dir="auto">model and wait-before-signal, we can't infer when something is about</div> <div dir="auto">to be done. Only the client knows when it submitted its last node in</div> <div dir="auto">the dependency graph and the whole mess is unblocked. Second, is that</div> <div dir="auto">the dma_fence is created within the client's driver context. If it's</div> <div dir="auto">created compositor-side, the kernel doesn't know who to blame if</div> <div dir="auto">things go badly. If we create it in the client, it's pretty easy to</div> <div dir="auto">make context death on -ETIME part of the contract.</div> <div dir="auto"><br></div> <div dir="auto">(Before danvet jumps in here and rants about how UMF -> dma_fence</div> <div dir="auto">isn't possible, I haven't forgotten. I'm pretending, for now, that</div> <div dir="auto">we've solved some of those problems.)</div> </blockquote> <div dir="auto"><br></div> <div dir="auto">Funny how we've come full circle to the original proposal here ...</div> <div dir="auto"><br></div> <div dir="auto">If we really want a kernel primitive for this - and I think it's a</div> <div dir="auto">good idea, since can help surface 'badness' in a way which is</div> <div dir="auto">observable by e.g. session managers in a way analogous to cgroup stats</div> <div dir="auto">and controls - how about this for a counter-proposal? Client exports a</div> <div dir="auto">FD for its context/queue and sends it to winsys as part of setup,</div> <div dir="auto">compositor can ioctl() on that to kill it, which lands in the same</div> <div dir="auto">zap/zap/zap/zap/ban codepath as GPU hangs do today. It's a bigger</div> <div dir="auto">hammer than per-sync-point primitives, but you as a client have to</div> <div dir="auto">accept the social contract that if you want to participate in a</div> <div dir="auto">session, your context has to be making forward progress and you aren't</div> <div dir="auto">writing cheques the compositor can't cash.</div></blockquote></div><div dir="auto"><br></div><div dir="auto">The compositor already has that. It can kick the client's Wayland protocol connection. Banning the context from the kernel might be nice too but kicking it is probably sufficient.</div><div dir="auto"><br></div><div dir="auto">Side-note to danvet: Do we need a plan for UMF with persistent contexts? My gut says that's a very bad idea but this made me think I should say least pose the question.</div><div dir="auto"><br></div><div id="aqm-original" style="color: black;" dir="auto"><blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #808080; padding-left: 0.75ex;"><div dir="auto"></div> <div dir="auto">I'm also going to pre-emptively agree with other-Dan; I'm extremely</div> <div dir="auto">wary of anything which tries to make UMF look even a little bit like</div> <div dir="auto">sync_file. The requirements to support them are so wildly different</div> <div dir="auto">that I'd almost rather a totally orthogonal interface so that there's</div> <div dir="auto">no danger of confusing the two. Us sophisticates on this thread can</div> <div dir="auto">eat the mild annoyance of typing out separate codepaths, but it's much</div> <div dir="auto">worse for anyone else who may look at a UMF wolf in dma_fence sheep's</div> <div dir="auto">clothing then only later be substantially more annoyed when they</div> <div dir="auto">realise that it's not anything like they thought it was.</div> <div dir="auto"><br></div> <div dir="auto">So let's keep sync_file for what it is, and for UMF since the usage is</div> <div dir="auto">so radically different, build out whatever we do around making the</div> <div dir="auto">uAPI as useful as possible for what we want to do with it. The real</div> <div dir="auto">complexity in handling the difference between UMF and 'real' fences is</div> <div dir="auto">in how they behave, not in how they look.</div></blockquote></div><div dir="auto"><br></div><div dir="auto">Sounds good.</div><div dir="auto"><br></div><div id="aqm-original" style="color: black;" dir="auto"><blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #808080; padding-left: 0.75ex;"><div dir="auto"></div> <blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #0099CC; padding-left: 0.75ex;"> <div dir="auto">Another option is to just stall on the UMF until it's done. Yeah,</div> <div dir="auto">kind-of terrible and high-latency, but it always works and doesn't</div> <div dir="auto">involve any complex logic to kill clients. If a client never gets</div> <div dir="auto">around to signaling a fence, it just never repaints. The compositor</div> <div dir="auto">keeps going like nothing's wrong. Maybe, if the client submits lots</div> <div dir="auto">of frames without ever triggering, it'll hit some max queue depth</div> <div dir="auto">somewhere and kill it but that's it. More likely, the client's</div> <div dir="auto">vkAcquireNextImage will start timing out and it'll crash.</div> <div dir="auto"><br></div> <div dir="auto">I suspect where we might actually land is some combination of the two</div> <div dir="auto">depending on client choice. If the client wants to be dumb, it gets</div> <div dir="auto">the high-latency always-works path. If the client really wants</div> <div dir="auto">lowest-latency VRR, it has to take the smarter path and risk</div> <div dir="auto">VK_ERROR_DEVICE_LOST if it misses too far.</div> </blockquote> <div dir="auto"><br></div> <div dir="auto">We already have to handle unresponsive clients. If your browser</div> <div dir="auto">livelocks today (say because it's Chrome and you hotunplug your</div> <div dir="auto">monitor at the wrong time with active media playback in an inactive</div> <div dir="auto">tab in an inactive window ... hypothetically), </div></blockquote></div><div dir="auto"><br></div><div dir="auto">That's an oddly specific hypothetical...</div><div dir="auto"><br></div><div id="aqm-original" style="color: black;" dir="auto"><blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #808080; padding-left: 0.75ex;"><div dir="auto">yourr Wayland server</div> <div dir="auto">notices that it isn't responding to pings, throws up the 'do you want</div> <div dir="auto">to force-quit?' dialog and kills the client; it's actually really</div> <div dir="auto">simple logic. So we just hook unsignaled fences up to the same. (And,</div> <div dir="auto">if we have the context-kill primitive, trigger that on our way out.)</div> <div dir="auto"><br></div> <div dir="auto">So yeah, we already have all the complexity points to put particular</div> <div dir="auto">surface trees in limbo (thanks to subsurface sync mode), we already</div> <div dir="auto">have all the complexity points to separate realised surface trees from</div> <div dir="auto">pixels on screen, and we already have the complexity points for</div> <div dir="auto">different parts of the surface trees being rendered at different</div> <div dir="auto">times. Checking on fence progression is just a little more typing</div> <div dir="auto">around those interface points which already exist, and zapping clients</div> <div dir="auto">is utterly trivial.</div></blockquote></div><div dir="auto"><br></div><div dir="auto">👍</div><div dir="auto"><br></div><div id="aqm-original" style="color: black;" dir="auto"><blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #808080; padding-left: 0.75ex;"><div dir="auto"></div> <blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #0099CC; padding-left: 0.75ex;"> <div dir="auto">But the point of all of this is, neither of the above two paths have</div> <div dir="auto">anything to do with the compositor calling a "wait for submit" ioctl.</div> <div dir="auto">Building a design around that and baking it into protocol is, IMO, a</div> <div dir="auto">mistake. I don't see any valid way to handle this mess without "wait</div> <div dir="auto">for sumbit" either not existing or existing only client-side for the</div> <div dir="auto">purposes of WSI.</div> </blockquote> <div dir="auto"><br></div> <div dir="auto">I'm still on the fence (sorry) about a wait-before-submit ioctl. For</div> <div dir="auto">the sync_file-based timeline syncobjs that we have today, yes it is</div> <div dir="auto">helpful, and we do already have it, it's just the wrong shape in being</div> <div dir="auto">sleep rather than epoll.</div></blockquote></div><div dir="auto"><br></div><div dir="auto">I still don't see why we're still talking about timeline syncobj...</div><div dir="auto"><br></div><div id="aqm-original" style="color: black;" dir="auto"><blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #808080; padding-left: 0.75ex;"><div dir="auto"></div> <div dir="auto">For UMF, taking it as a given that the kernel really has no visibility</div> <div dir="auto">at all into syncpoint progress, then the kernel is conceptually a</div> <div dir="auto">worse place to spin-sleep than userspace is, because why account the</div> <div dir="auto">CPU burn to a kthread rather than a real PID, and lose</div> <div dir="auto">latency/efficiency on context switches when you do wake up?</div> <div dir="auto"><br></div> <div dir="auto">But also, the kernel is conceptually the best place to spin-sleep,</div> <div dir="auto">because it can fuse waits and do better wakeup quantisation than</div> <div dir="auto">userspace can. And I'm still hopeful that the IHVs and Windows can</div> <div dir="auto">both step away from the postmodern 'synchronisation doesn't mean</div> <div dir="auto">anything anymore, just poll in a lap-burning loop' approach that we've</div> <div dir="auto">been presented (sorry) with, where we at least get doorbells which</div> <div dir="auto">allow the kernel to do polling much smarter than quantising timers</div> <div dir="auto">('this fence might not have triggered yet, but _something_ happened</div> <div dir="auto">which might have triggered it so why not check?').</div></blockquote></div><div dir="auto"><br></div><div dir="auto">I think we can and do do something better than just poll on the memory. I'm not sure on the details but I've been told that we can set some sort of interrupt-like thing on the address so it's not actually a spin. Even without that, done hardware has some way that a command buffer can trigger an interrupt. If the protocol is to write memory and then trigger an interrupt rather than just write memory, that gives us something if a doorbell. Not as convenient, maybe, but it'd help with power consumption, etc.</div><div dir="auto"><br></div><div dir="auto">--Jason</div><div dir="auto"><br></div><div id="aqm-original" style="color: black;" dir="auto"><blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #808080; padding-left: 0.75ex;"><div dir="auto"></div> <div dir="auto">On the other other hand, the only winsys case for burning poll in a</div> <div dir="auto">tight loop is flipping as quickly as possible straight to a VRR</div> <div dir="auto">display. In that case, you're definitely running on mains power so</div> <div dir="auto">you're just melting the polar ice caps rather than your lap, and</div> <div dir="auto">you've got everything fully lit up anyway so the power cost of polling</div> <div dir="auto">is immaterial. For FRR, the compositor already has a fixed deadline at</div> <div dir="auto">which it will wake up and make a hard binding decision about which</div> <div dir="auto">image to present - this includes XR as well. So we don't have to worry</div> <div dir="auto">about optimising a polling loop, because there isn't one: we wake up</div> <div dir="auto">once, we check once, and if the client's missed then too bad, try</div> <div dir="auto">again next frame.</div> <div dir="auto"><br></div> <div dir="auto">As you can see, much like userspace memory fences, my position on</div> <div dir="auto">which way to go here is not knowable upfront, and depends on when</div> <div dir="auto">exactly you observe it. Hopefully someone can come back with an</div> <div dir="auto">argument compelling enough either way that I have something better to</div> <div dir="auto">do than to try to pun my way out of having more hands than Ganesh. I</div> <div dir="auto">don't think it's material to the design or implementation of winsys</div> <div dir="auto">support though.</div> <div dir="auto"><br></div> <div dir="auto">Cheers,</div> <div dir="auto">Daniel</div> </blockquote> </div><div dir="auto"><br></div> </div></body> </html>