<html><head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> </head> <body text="#000000" bgcolor="#FFFFFF"> <div class="moz-cite-prefix">Monk needs to answer this, but I don't think that this will work. This explanation even sounds like only one VF can use the feature at the same time, is that correct? Regards, Christian. Am 20.04.20 um 14:08 schrieb Tao, Yintian: </div> <blockquote type="cite" cite="mid:MN2PR12MB3039D470FCE5F13DF9AD9945E5D40@MN2PR12MB3039.namprd12.prod.outlook.com"> <meta name="Generator" content="Microsoft Word 15 (filtered medium)">  <style></style> <div class="WordSection1"> Hi Monk, Christian <o:p></o:p> <o:p> </o:p> <o:p> </o:p> According to the discussion with Jacob offline, UMD will only enable SPM feature when testing RGP.<o:p></o:p> And under virtualization , only pp_one_vf mode can test RGP.<o:p></o:p> Therefore, whether we can directly use MMIO to READ/WRITE register for RLC_SPM_MC_CNTL?<o:p></o:p> <o:p> </o:p> <o:p> </o:p> Best Regards<o:p></o:p> Yintian Tao<o:p></o:p> <o:p> </o:p> <div> <div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm"> From: amd-gfx <a class="moz-txt-link-rfc2396E" href="mailto:amd-gfx-bounces@lists.freedesktop.org"><amd-gfx-bounces@lists.freedesktop.org></a> On Behalf Of Liu, Monk Sent: 2020年4月20日 16:33 To: Koenig, Christian <a class="moz-txt-link-rfc2396E" href="mailto:Christian.Koenig@amd.com"><Christian.Koenig@amd.com></a>; He, Jacob <a class="moz-txt-link-rfc2396E" href="mailto:Jacob.He@amd.com"><Jacob.He@amd.com></a> Cc: <a class="moz-txt-link-abbreviated" href="mailto:amd-gfx@lists.freedesktop.org">amd-gfx@lists.freedesktop.org</a> Subject: RE: why we need to do infinite RLC_SPM register setting during VM flush<o:p></o:p> </div> </div> <o:p> </o:p> Christian<o:p></o:p> <o:p></o:p> What we want to do is like:<o:p></o:p> Read reg value from RLC_SPM_MC_CNTL to tmp<o:p></o:p> Set bits:3:0 to VMID to tmp<o:p></o:p> Write tmp to RLC_SPM_MC_CNTL<o:p></o:p> <o:p></o:p> I didn’t find any PM4 packet on GFX9/10 can achieve above goal …. <o:p></o:p> <o:p></o:p> <o:p></o:p> <div> _____________________________________<o:p></o:p> Monk Liu|GPU Virtualization Team |AMD<o:p></o:p> <img style="width:.8333in;height:.8333in" id="_x0000_i1025" src="cid:part1.BF021B7C.4820FFBB@amd.com" alt="sig-cloud-gpu" class="" width="80" height="80"><o:p></o:p> </div> <o:p></o:p> <div> <div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm"> From: Christian König <<a href="mailto:ckoenig.leichtzumerken@gmail.com" moz-do-not-send="true">ckoenig.leichtzumerken@gmail.com</a>> Sent: Monday, April 20, 2020 4:03 PM To: Liu, Monk <<a href="mailto:Monk.Liu@amd.com" moz-do-not-send="true">Monk.Liu@amd.com</a>>; He, Jacob <<a href="mailto:Jacob.He@amd.com" moz-do-not-send="true">Jacob.He@amd.com</a>>; Koenig, Christian <<a href="mailto:Christian.Koenig@amd.com" moz-do-not-send="true">Christian.Koenig@amd.com</a>> Cc: <a href="mailto:amd-gfx@lists.freedesktop.org" moz-do-not-send="true">amd-gfx@lists.freedesktop.org</a> Subject: Re: why we need to do infinite RLC_SPM register setting during VM flush<o:p></o:p> </div> </div> <o:p></o:p> <div> I would also prefer to update the SPM VMID register using PM4 packets instead of the current handling. Regards, Christian. Am 20.04.20 um 09:50 schrieb Liu, Monk:<o:p></o:p> </div> <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"> I just try to explain what I want to do here, no real patch formalized yet <o:p></o:p> <o:p></o:p> <div> _____________________________________<o:p></o:p> Monk Liu|GPU Virtualization Team |AMD<o:p></o:p> <img style="width:.8333in;height:.8333in" id="Picture_x0020_1" src="cid:part1.BF021B7C.4820FFBB@amd.com" alt="sig-cloud-gpu" class="" width="80" height="80" border="0"><o:p></o:p> </div> <o:p></o:p> <div> <div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm"> From: He, Jacob <a href="mailto:Jacob.He@amd.com" moz-do-not-send="true"><Jacob.He@amd.com></a> Sent: Monday, April 20, 2020 3:45 PM To: Liu, Monk <a href="mailto:Monk.Liu@amd.com" moz-do-not-send="true"><Monk.Liu@amd.com></a>; Koenig, Christian <a href="mailto:Christian.Koenig@amd.com" moz-do-not-send="true"><Christian.Koenig@amd.com></a> Cc: <a href="mailto:amd-gfx@lists.freedesktop.org" moz-do-not-send="true">amd-gfx@lists.freedesktop.org</a> Subject: Re: why we need to do infinite RLC_SPM register setting during VM flush<o:p></o:p> </div> </div> <o:p></o:p> [AMD Official Use Only - Internal Distribution Only]<o:p></o:p> <o:p></o:p> <div> <div> Do you miss a file which adds spm_updated to vm structure?<o:p></o:p> </div> <div class="MsoNormal" style="text-align:center" align="center"> <hr width="98%" size="2" align="center"> </div> <div id="divRplyFwdMsg"> From: Liu, Monk <<a href="mailto:Monk.Liu@amd.com" moz-do-not-send="true">Monk.Liu@amd.com</a>> Sent: Monday, April 20, 2020 3:32 PM To: He, Jacob <<a href="mailto:Jacob.He@amd.com" moz-do-not-send="true">Jacob.He@amd.com</a>>; Koenig, Christian <<a href="mailto:Christian.Koenig@amd.com" moz-do-not-send="true">Christian.Koenig@amd.com</a>> Cc: <a href="mailto:amd-gfx@lists.freedesktop.org" moz-do-not-send="true">amd-gfx@lists.freedesktop.org</a> <<a href="mailto:amd-gfx@lists.freedesktop.org" moz-do-not-send="true">amd-gfx@lists.freedesktop.org</a>> Subject: why we need to do infinite RLC_SPM register setting during VM flush <o:p></o:p> <div> <o:p></o:p> </div> </div> <div> <div> Hi Jaco & Christian<o:p></o:p> <o:p></o:p> As titled , check below patch:<o:p></o:p> <o:p></o:p> commit 10790a09ea584cc832353a5c2a481012e5e31a13<o:p></o:p> Author: Jacob He <<a href="mailto:jacob.he@amd.com" moz-do-not-send="true">jacob.he@amd.com</a>><o:p></o:p> Date: Fri Feb 28 20:24:41 2020 +0800<o:p></o:p> <o:p></o:p> drm/amdgpu: Update SPM_VMID with the job's vmid when application reserves the vmid<o:p></o:p> <o:p></o:p> SPM access the video memory according to SPM_VMID. It should be updated<o:p></o:p> with the job's vmid right before the job is scheduled. SPM_VMID is a<o:p></o:p> global resource<o:p></o:p> <o:p></o:p> Change-Id: Id3881908960398f87e7c95026a54ff83ff826700<o:p></o:p> Signed-off-by: Jacob He <<a href="mailto:jacob.he@amd.com" moz-do-not-send="true">jacob.he@amd.com</a>><o:p></o:p> Reviewed-by: Christian König <<a href="mailto:christian.koenig@amd.com" moz-do-not-send="true">christian.koenig@amd.com</a>><o:p></o:p> <o:p></o:p> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c<o:p></o:p> index 6e6fc8c..ba2236a 100644<o:p></o:p> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c<o:p></o:p> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c<o:p></o:p> @@ -1056,8 +1056,12 @@ int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job,<o:p></o:p> struct dma_fence *fence = NULL;<o:p></o:p> bool pasid_mapping_needed = false;<o:p></o:p> unsigned patch_offset = 0;<o:p></o:p> + bool update_spm_vmid_needed = (job->vm && (job->vm->reserved_vmid[vmhub] != NULL));<o:p></o:p> int r;<o:p></o:p> <o:p></o:p> + if (update_spm_vmid_needed && adev->gfx.rlc.funcs->update_spm_vmid)<o:p></o:p> + adev->gfx.rlc.funcs->update_spm_vmid(adev, job->vmid);<o:p></o:p> +<o:p></o:p> if (amdgpu_vmid_had_gpu_reset(adev, id)) {<o:p></o:p> gds_switch_needed = true;<o:p></o:p> vm_flush_needed = true;<o:p></o:p> <o:p></o:p> this update_spm_vmid() looks an completely overkill to me, we only need to do it once for its VM …<o:p></o:p> <o:p></o:p> in SRIOV the register reading/writing for update_spm_vmid() is now carried by KIQ thus there is too much burden on KIQ for such unnecessary jobs …<o:p></o:p> <o:p></o:p> I want to change it to only do it once per VM, like:<o:p></o:p> <o:p></o:p> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c<o:p></o:p> index 6e6fc8c..ba2236a 100644<o:p></o:p> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c<o:p></o:p> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c<o:p></o:p> @@ -1056,8 +1056,12 @@ int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job,<o:p></o:p> struct dma_fence *fence = NULL;<o:p></o:p> bool pasid_mapping_needed = false;<o:p></o:p> unsigned patch_offset = 0;<o:p></o:p> + bool update_spm_vmid_needed = (job->vm && (job->vm->reserved_vmid[vmhub] != NULL));<o:p></o:p> int r;<o:p></o:p> <o:p></o:p> + if (update_spm_vmid_needed && adev->gfx.rlc.funcs->update_spm_vmid && !vm->spm_updated) {<o:p></o:p> + adev->gfx.rlc.funcs->update_spm_vmid(adev, job->vmid); <o:p></o:p> + vm->spm_updated = true;<o:p></o:p> + }<o:p></o:p> <o:p></o:p> if (amdgpu_vmid_had_gpu_reset(adev, id)) {<o:p></o:p> gds_switch_needed = true;<o:p></o:p> vm_flush_needed = true;<o:p></o:p> <o:p></o:p> what do you think ?<o:p></o:p> <o:p></o:p> P.S.: the best way is to let GFX ring itself to do the update_spm_vmid() instead of let CPU doing it, e.g.: we put more PM4 command in VM-FLUSH packets …. <o:p></o:p> But I prefer the simple way first like I demonstrated above <o:p></o:p> _____________________________________<o:p></o:p> Monk Liu|GPU Virtualization Team |AMD<o:p></o:p> <img style="width:.8333in;height:.8333in" id="x_Picture_x0020_1" src="cid:part1.BF021B7C.4820FFBB@amd.com" alt="sig-cloud-gpu" class="" width="80" height="80" border="0"><o:p></o:p> <o:p></o:p> </div> </div> </div> <o:p></o:p> <pre>_______________________________________________<o:p></o:p></pre> <pre>amd-gfx mailing list<o:p></o:p></pre> <pre><a href="mailto:amd-gfx@lists.freedesktop.org" moz-do-not-send="true">amd-gfx@lists.freedesktop.org</a><o:p></o:p></pre> <pre><a href="https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&data=02%7C01%7Cyintian.tao%40amd.com%7C94ed105b251d4c2629cc08d7e5056f39%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637229683834893085&sdata=IJiVlGsX0rL0%2FcTvlSWquG%2F3aufz9R%2FvMlkk%2BpVaQSM%3D&reserved=0" moz-do-not-send="true">https://lists.freedesktop.org/mailman/listinfo/amd-gfx</a><o:p></o:p></pre> </blockquote> <o:p></o:p> </div> </blockquote> </body> </html>