<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
Well that's the whole reason I'm asking :)<br>
<br>
Why do you think it should be added as dependency in
amdgpu_vm_sdma_update? As far as I can see that is complete
nonsense.<br>
<br>
Page table updates never depend on TLB flushes, it's the TLB flush
which depends on the page table update.<br>
<br>
Regards,<br>
Christian.<br>
<br>
<div class="moz-cite-prefix">Am 06.09.24 um 16:10 schrieb
Andjelkovic, Dejan:<br>
</div>
<blockquote type="cite" cite="mid:CO6PR12MB5441B3A86656FFD6D7EE9300E79E2@CO6PR12MB5441.namprd12.prod.outlook.com">
<style type="text/css" style="display:none;">P {margin-top:0;margin-bottom:0;}</style>
<p style="font-family:Calibri;font-size:10pt;color:#0000FF;margin:5pt;font-style:normal;font-weight:normal;text-decoration:none;" align="Left">
[AMD Official Use Only - AMD Internal Distribution Only]<br>
</p>
<br>
<div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I might have worded that poorly, I meant that it seems like
TLB flush is out of sync with the SDMA update, which leads to
a page fault reliably. I don't feel it has anything to do with
the implicit sync in itself. When TLB fence is created it's
added to the dma_resv of the vm's root buffer object with
BOOKKEEP usage specified, in order to make sure no PD/PT is
freed before the flush. But I don't think it's being added as
a job dependency within the amdgpu_vm_sdma_update, we're
adding all the fences found within the dma_resv object with
KERNEL usage specified. I may be missing something so I'd love
to hear what you think.<br>
<br>
Best regards,</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Dejan</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b>
Koenig, Christian <a class="moz-txt-link-rfc2396E" href="mailto:Christian.Koenig@amd.com"><Christian.Koenig@amd.com></a><br>
<b>Sent:</b> Thursday, September 5, 2024 2:40 PM<br>
<b>To:</b> Andjelkovic, Dejan
<a class="moz-txt-link-rfc2396E" href="mailto:Dejan.Andjelkovic@amd.com"><Dejan.Andjelkovic@amd.com></a>;
<a class="moz-txt-link-abbreviated" href="mailto:amd-gfx@lists.freedesktop.org">amd-gfx@lists.freedesktop.org</a>
<a class="moz-txt-link-rfc2396E" href="mailto:amd-gfx@lists.freedesktop.org"><amd-gfx@lists.freedesktop.org></a><br>
<b>Cc:</b> Prica, Nikola <a class="moz-txt-link-rfc2396E" href="mailto:Nikola.Prica@amd.com"><Nikola.Prica@amd.com></a>;
Kuehling, Felix <a class="moz-txt-link-rfc2396E" href="mailto:Felix.Kuehling@amd.com"><Felix.Kuehling@amd.com></a>; Deng, Emily
<a class="moz-txt-link-rfc2396E" href="mailto:Emily.Deng@amd.com"><Emily.Deng@amd.com></a><br>
<b>Subject:</b> Re: [PATCH] drm/amdgpu: Raise dma resv usage
for created TLB fence</font>
<div> </div>
</div>
<div>Well that explanation doesn't seem to make much sense
either.<br>
<br>
What do you mean with TLB flush is occurring prematurely?<br>
<br>
Regards,<br>
Christian.<br>
<br>
<div class="x_moz-cite-prefix">Am 05.09.24 um 14:38 schrieb
Andjelkovic, Dejan:<br>
</div>
<blockquote type="cite">
<style type="text/css" style="display:none">p
{margin-top:0;
margin-bottom:0}</style>
<p style="font-family:Calibri; font-size:10pt; color:#0000FF; margin:5pt; font-style:normal; font-weight:normal; text-decoration:none" align="Left">
[AMD Official Use Only - AMD Internal Distribution Only]<br>
</p>
<br>
<div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Hi there. We're running into a page fault issue that's
very easily reproducible on a SRIOV environment when
using SDMA for page table updates. Going through mapping
logs and trace files, it seems TLB flush is occurring
prematurely. Changing the usage to KERNEL completely
stops the page fault from occurring with no performance
impact, which was confirmed with extensive testing.
Looking through amdgpu_vm_sdma.c, namely within
amdgpu_vm_sdma_update when waiting for PD/PT moves to be
completed, fences are iterated with KERNEL usage
specified which are then added as a job dependency.</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Best regards,</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Dejan</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Koenig, Christian
<a class="x_moz-txt-link-rfc2396E" href="mailto:Christian.Koenig@amd.com" moz-do-not-send="true"><Christian.Koenig@amd.com></a><br>
<b>Sent:</b> Thursday, September 5, 2024 1:17 PM<br>
<b>To:</b> Andjelkovic, Dejan <a class="x_moz-txt-link-rfc2396E" href="mailto:Dejan.Andjelkovic@amd.com" moz-do-not-send="true">
<Dejan.Andjelkovic@amd.com></a>; <a class="x_moz-txt-link-abbreviated moz-txt-link-freetext" href="mailto:amd-gfx@lists.freedesktop.org" moz-do-not-send="true">
amd-gfx@lists.freedesktop.org</a> <a class="x_moz-txt-link-rfc2396E" href="mailto:amd-gfx@lists.freedesktop.org" moz-do-not-send="true">
<amd-gfx@lists.freedesktop.org></a><br>
<b>Cc:</b> Prica, Nikola <a class="x_moz-txt-link-rfc2396E" href="mailto:Nikola.Prica@amd.com" moz-do-not-send="true">
<Nikola.Prica@amd.com></a>; Kuehling, Felix <a class="x_moz-txt-link-rfc2396E" href="mailto:Felix.Kuehling@amd.com" moz-do-not-send="true">
<Felix.Kuehling@amd.com></a>; Deng, Emily <a class="x_moz-txt-link-rfc2396E" href="mailto:Emily.Deng@amd.com" moz-do-not-send="true">
<Emily.Deng@amd.com></a><br>
<b>Subject:</b> Re: [PATCH] drm/amdgpu: Raise dma resv
usage for created TLB fence</font>
<div> </div>
</div>
<div class="x_BodyFragment"><font size="2"><span style="font-size:11pt">
<div class="x_PlainText">Am 05.09.24 um 10:58
schrieb Dejan Andjelkovic:<br>
> When using SDMA for PT updates, a TLB fence
hooked to a buffer<br>
> objects dma resv object with usage declared
as BOOKKEEP leaves a<br>
> chance for TLB flush to occur prematurely.
This will lead to a page<br>
> fault. Raising the usage from BOOKKEEP to
KERNEL removes this<br>
> possibility.<br>
<br>
Well that's complete nonsense. The usage model is
for implicit syncing <br>
and not even remotely related to TLB flushing.<br>
<br>
What exactly is the problem you run into?<br>
<br>
Regards,<br>
Christian.<br>
<br>
><br>
> Signed-off-by: Dejan Andjelkovic <a class="x_moz-txt-link-rfc2396E" href="mailto:Dejan.Andjelkovic@amd.com" moz-do-not-send="true">
<Dejan.Andjelkovic@amd.com></a><br>
> ---<br>
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2
+-<br>
> 1 file changed, 1 insertion(+), 1
deletion(-)<br>
><br>
> diff --git
a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c<br>
> index f93804902fd3..8efc2cf9bbb0 100644<br>
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c<br>
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c<br>
> @@ -928,7 +928,7 @@
amdgpu_vm_tlb_flush(struct amdgpu_vm_update_params
*params,<br>
> <br>
> /* Makes sure no PD/PT is
freed before the flush */<br>
>
dma_resv_add_fence(vm->root.bo->tbo.base.resv,
*fence,<br>
> -
DMA_RESV_USAGE_BOOKKEEP);<br>
> +
DMA_RESV_USAGE_KERNEL);<br>
> }<br>
> }<br>
> <br>
<br>
</div>
</span></font></div>
</div>
</blockquote>
<br>
</div>
</div>
</blockquote>
<br>
</body>
</html>