<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Apr 8, 2021 at 2:27 AM Gerd Hoffmann <<a href="mailto:kraxel@redhat.com" target="_blank">kraxel@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">> > +<br>
> > + if (vgdev->has_resource_blob) {<br>
> > + params.blob_mem = VIRTGPU_BLOB_MEM_GUEST;<br>
> > + params.blob_flags = VIRTGPU_BLOB_FLAG_USE_SHAREABLE;<br>
> ><br>
> <br>
> This creates some log spam with crosvm + virgl_3d + vanilla linux, since<br>
> transfers don't work for guest blobs. Two options:<br>
> <br>
> a) Add vgdev->has_virgl_3d check and don't create a guest blob in that case.<br>
> b) The interactions between TRANSFER_TO_HOST_2D and VIRTGPU_BLOB_MEM_GUEST<br>
> are a bit under-defined in the spec.<br>
<br>
Indeed.<br>
<br>
> Though since you don't have a host<br>
> side resource, you can safely skip the transfer and crosvm can be modified<br>
> to do the right thing in case of RESOURCE_FLUSH.<br>
<br>
IIRC the VIRTGPU_BLOB_FLAG_USE_SHAREABLE flag means that the host *can*<br>
create a shared mapping (i.e. the host seeing guest-side changes without<br>
explicit transfer doesn't cause problems for the guest). It doesn not<br>
mean the host *must* create a shared mapping (note that there is no<br>
negotiation whenever the host supports shared mappings or not).<br></blockquote><div><br></div><div><div>VIRTGPU_BLOB_FLAG_USE_SHAREABLE means guest userspace intends to share the blob resource with another virtgpu driver instance via drmPrimeHandleToFd. It's a rough analogue to VkExportMemoryAllocateInfoKHR or PIPE_BIND_USE_SHARED.<br></div><div><br></div><div><div>The dumb case is a bit interesting because there is no userspace to provide that information. Though I think even VIRTGPU_BLOB_FLAG_USE_MAPPABLE is fine, since for my vanilla Linux setup, I'm seeing the guest blob is mapped only and drmPrimeHandleToFd(..) isn't called on it. We can also modify the virtio-gpu spec to say "blob_flags may be undefined/zero for BLOB_MEM_GUEST when 3D mode is not on".</div><div><br></div><div><div>Though all options work for me. The implicit dumb blob udmabuf case for me is more about advancing blob development rather than being super rigorous.</div><div></div></div><div></div></div><div></div></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
So the transfer calls are still needed, and the host can decide to<br>
shortcut them in case it can create a shared mapping. In case there is<br>
no shared mapping (say due to missing udmabuf support) the host can<br>
fallback to copying.<br></blockquote><div><br></div><div>Transfers are a bit under-defined for BLOB_MEM_GUEST. Even without udmabuf on the host, there is no host side resource for guest-only blobs? Before blob resources, the dumb flow was:</div><div><br></div><div><div>1) update guest side resource</div><div>2) TRANSFER_TO_HOST_2D to copy guest side contents to host side private resource [Pixman??]</div><div>3) RESOURCE_FLUSH to copy the host-side contents to the framebuffer and page-flip</div><div><br></div><div>At least for crosvm, this is possible:</div><div><br></div><div><div>1) update guest side resource</div><div>2) RESOURCE_FLUSH to copy the guest-side contents to the framebuffer and pageflip<br></div><div><br></div><div>With implicit udmabuf, it may be possible to do this:</div><div><br></div><div><div>1) update guest side resource</div><div>2) RESOURCE_FLUSH to page-flip</div></div></div></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
So I think crosvm should be fixed to not consider transfer commands for<br>
VIRTGPU_BLOB_MEM_GUEST resources an error.<br></blockquote><div><br></div><div>It's a simple change to make and we can definitely do it, if TRANSFER_2D is helpful for the QEMU case. I haven't looked at the QEMU side patches.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
> It makes a ton of sense to have a explicit udmabuf-like flag<br>
> ("BLOB_FLAG_CREATE_GUEST_HANDLE" or "BLOB_FLAG_HANDLE_FROM_GUEST" -- want<br>
> to host OS agnostic -- any other ideas?), especially with 3d mode.<br>
<br>
Why? Can't this be simply an host implementation detail which the guest<br>
doesn't need to worry about?<br></blockquote><div><br></div><div><div>For 3D mode, it's desirable to create an {EGL image}/{VkDeviceMemory} from guest memory for certain zero-copy use cases. If no explicit guarantee exists for the paravirtualized user-space that there will be a host side OS-specific handle associated with guest memory, then guest user space must fall-back to old-style transfers.</div><div><br></div><div>For the PCI-passthrough + guest blob case, the end goal is to share it with the host compositor. If there is no guarantee the guest memory can be converted to an OS-handle (to share with the host compositor), then I think the guest user space should fallback to another technique involving memcpy() to share the memory.</div><div><br></div><div><div>So essentially, thinking for two new protocol additions:</div><div><br></div><div>F_CREATE_GUEST_HANDLE (or F_HANDLE_FROM_GUEST) --> means an OS-specific udmabuf-like mechanism exists on the host.<br></div><div><br></div><div>BLOB_FLAG_CREATE_GUEST_HANDLE (or BLOB_FLAG_HANDLE_FROM_GUEST)--> tells host userspace "you must create a udmabuf" [or OS-specific equivalent] upon success</div></div><div><br></div><div>Though much testing/work remains (both with the PCI passthough case + virgl3d case), could be a good chance to float the nomenclature by everyone. Happy to collaborate further with Tina/Vivek on making such a thing happen.</div></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
take care,<br>
Gerd<br>
<br>
</blockquote></div></div>