<html dir="ltr"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> </head> <body style="text-align:left; direction:ltr;"> <div>On Thu, 2022-03-24 at 11:56 +0200, Pekka Paalanen wrote:</div> <blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"> <pre>On Wed, 23 Mar 2022 14:19:08 +0000</pre> <pre>"Hoosier, Matt" <</pre> <a href="mailto:Matt.Hoosier@garmin.com"> <pre>Matt.Hoosier@garmin.com</pre> </a> <pre>> wrote:</pre> <pre><br></pre> <blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"> <pre>Hi,</pre> <pre><br></pre> <pre>I recently had a reason to wade through Mutter's code to support</pre> <pre>systems with more than one GPU. I was a bit surprised to see that</pre> <pre>it supports several different strategies for dealing with</pre> <pre>scanning out a buffer on a KMS output not associated with the GPU</pre> <pre>where the buffer was originally rendered.</pre> </blockquote> <pre><br></pre> <pre>Hi,</pre> <pre><br></pre> <pre>indeed. The reason for multiple paths is that different systems</pre> <pre>don't support all ways, or do support some of the ways but the</pre> <pre>performance might be abysmal. Knowing which path to take is a</pre> <pre>difficult problem, and other than benchmarking (which I didn't</pre> <pre>implement in Mutter) you can't really know if what you picked is</pre> <pre>going to be fine.</pre> <pre><br></pre> <blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"> <pre>In particular, the approach of using the secondary GPU's OpenGL</pre> <pre>implementation to blit into a dumb buffer was really unexpected.</pre> <pre>Typically, dumb buffers get described as a really slow, uncached</pre> <pre>mapping of GPU memory into the CPU.</pre> <pre><br></pre> <pre>The support got added here (by Pekka):</pre> <pre><br></pre> <a href="https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/615"> <pre>https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/615</pre> </a> <pre><br></pre> </blockquote> <pre><br></pre> <pre>That MR is using the primary GPU to blit, not the secondary GPU.</pre> <pre><br></pre> <pre>If a secondary GPU can have a hardware accelerated OpenGL context,</pre> <pre>I don't know why anyone would deliberately use dumb buffers on that</pre> <pre>device with OpenGL. GBM offers better ways.</pre> <pre><br></pre> <pre><br></pre> <pre>This MR cover letter has a better overview of all the methods:</pre> <a href="https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/810"> <pre></a><a href="https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/810">https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/810</a></pre> </blockquote> <div><br> </div> <div>Ah, even nicer. Thanks!</div> <div><br> </div> <div>In the ranked-order list of strategies there, the zero-copy technique is less preferred than the secondary GPU copy technique. Seems like you'd rarely ever fall through to the zero-copy strategy even if the GPU drivers do both support it. Anything subtle going on there that's good to be aware of? Like maybe a given driver typically supports secondary-GPU-copy XOR zero-copy, so you're fairly likely to reach the second strategy on systems that can handle it.</div> <div><br> </div> <blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"> <pre><br></pre> <pre><br></pre> <blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"> <pre>If I follow this right, the blit occurs directly between video</pre> <pre>memory owned by the primary GPU into dumb-buffer memory owned by</pre> <pre>the secondary GPU, without laboriously using the CPU to do PIO.</pre> </blockquote> <pre><br></pre> <pre>Correct.</pre> <pre><br></pre> <blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"> <pre>Does this imply that the two GPUs' drivers have to be at least</pre> <pre>minimally aware of each other to negotiate some kind of DMA path</pre> <pre>directly between the two?</pre> </blockquote> <pre><br></pre> <pre>I don't know the details. It depends on whether you can map</pre> <pre>secondary GPU memory to be written by the primary GPU. The specific</pre> <pre>use case here is iGPU as primary and virtual as secondary, which</pre> <pre>means that video memory for both is more or less "system RAM". No</pre> <pre>discrete VRAM involved.</pre> </blockquote> <div><br> </div> <div>Oh interesting. I hadn't realized that on the hybrid GPU systems even the dGPU uses system RAM. But on thinking about it, that's probably the only efficient way for the hardware to be designed.</div> <div><br> </div> <blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"> <pre><br></pre> <pre>It is accomplished through the kernel dmabuf framework where</pre> <pre>drivers export and import dmabuf.</pre> </blockquote> <div><br> </div> <div>Right, makes sense.</div> <div><br> </div> <div>So I wonder how I should reason about a system that's configured with 2x of the same discrete graphics card (AMD, if it matters). The compositor would arbitrarly pick whichever of those happened to enumerate first as the primary, and then it's down to the driver details as to which of the four migration paths gets chosen? For the moment, let's assume that none of the stock applications is bothering to use any sort of advanced dmabuf hinting to pick the right GPU node to correspond to the output on which it will eventually display.</div> <div><br> </div> <div>Thanks!</div> <div>Matt</div> <br> <hr> <font face="Arial" color="Gray" size="1"><br> CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of the intended recipient(s) and contain information that may be Garmin confidential and/or Garmin legally privileged. If you have received this email in error, please notify the sender by reply email and delete the message. Any disclosure, copying, distribution or use of this communication (including attachments) by someone other than the intended recipient is prohibited. Thank you.<br> </font> </body> </html>