<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> <meta name="Generator" content="Microsoft Exchange Server"> <style></style> </head> <body> <div> <div dir="auto">Hi Lionel, <div dir="auto"><br> </div> <div dir="auto">Well could you describe once more what the problem is?</div> <div dir="auto"><br> </div> <div dir="auto">Cause I don't fully understand why a rather normal tandem submission with two semaphores should fail in any way.</div> <div dir="auto"><br> </div> <div dir="auto">Regards,</div> <div dir="auto">Christian.</div> </div> <div class="x_gmail_extra"><br> <div class="x_gmail_quote">Am 02.08.2019 06:28 schrieb Lionel Landwerlin <lionel.g.landwerlin@intel.com>:<br type="attribution"> </div> </div> </div> <font size="2"><span style="font-size:11pt;"> <div class="PlainText">There aren't CTS tests covering the issue I was mentioning.<br> But we could add them.<br> <br> I don't have all the details regarding your implementation but even with <br> the "semaphore thread", I could see it running into the same issues.<br> What if a mix of binary & timeline semaphores are handed to vkQueueSubmit()?<br> <br> For example with queueA & queueB from 2 different VkDevice :<br> vkQueueSubmit(queueA, signal semA);<br> vkQueueSubmit(queueA, wait on [semA, timelineSemB]); with <br> timelineSemB triggering a wait before signal.<br> vkQueueSubmit(queueB, signal semA);<br> <br> <br> -Lionel<br> <br> On 02/08/2019 06:18, Zhou, David(ChunMing) wrote:<br> > Hi Lionel,<br> ><br> > By the Queue thread is a heavy thread, which is always resident in driver during application running, our guys don't like that. So we switch to Semaphore Thread, only when waitBeforeSignal of timeline happens, we spawn a thread to handle that wait. So we don't have your this issue.<br> > By the way, I already pass all your CTS cases for now. I suggest you to switch to Semaphore Thread instead of Queue Thread as well. It works very well.<br> ><br> > -David<br> ><br> > -----Original Message-----<br> > From: Lionel Landwerlin <lionel.g.landwerlin@intel.com><br> > Sent: Friday, August 2, 2019 4:52 AM<br> > To: dri-devel <dri-devel@lists.freedesktop.org>; Koenig, Christian <Christian.Koenig@amd.com>; Zhou, David(ChunMing) <David1.Zhou@amd.com>; Jason Ekstrand <jason@jlekstrand.net><br> > Subject: Threaded submission & semaphore sharing<br> ><br> > Hi Christian, David,<br> ><br> > Sorry to report this so late in the process, but I think we found an issue not directly related to syncobj timelines themselves but with a side effect of the threaded submissions.<br> ><br> > Essentially we're failing a test in crucible :<br> > func.sync.semaphore-fd.opaque-fd<br> > This test create a single binary semaphore, shares it between 2 VkDevice/VkQueue.<br> > Then in a loop it proceeds to submit workload alternating between the 2 VkQueue with one submit depending on the other.<br> > It does so by waiting on the VkSemaphore signaled in the previous iteration and resignaling it.<br> ><br> > The problem for us is that once things are dispatched to the submission thread, the ordering of the submission is lost.<br> > Because we have 2 devices and they both have their own submission thread.<br> ><br> > Jason suggested that we reestablish the ordering by having semaphores/syncobjs carry an additional uint64_t payload.<br> > This 64bit integer would represent be an identifier that submission threads will WAIT_FOR_AVAILABLE on.<br> ><br> > The scenario would look like this :<br> > - vkQueueSubmit(queueA, signal on semA);<br> > - in the caller thread, this would increment the syncobj additional u64 payload and return it to userspace.<br> > - at some point the submission thread of queueA submits the workload and signal the syncobj of semA with value returned in the caller thread of vkQueueSubmit().<br> > - vkQueueSubmit(queueB, wait on semA);<br> > - in the caller thread, this would read the syncobj additional<br> > u64 payload<br> > - at some point the submission thread of queueB will try to submit the work, but first it will WAIT_FOR_AVAILABLE the u64 value returned in the step above<br> ><br> > Because we want the binary semaphores to be shared across processes and would like this to remain a single FD, the simplest location to store this additional u64 payload would be the DRM syncobj.<br> > It would need an additional ioctl to read & increment the value.<br> ><br> > What do you think?<br> ><br> > -Lionel<br> <br> <br> </div> </span></font> </body> </html>