<html><head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> </head> <body> <div class="moz-cite-prefix">On 2021-11-10 5:15 a.m., Christian König wrote: </div> <blockquote type="cite" cite="mid:1862b795-3401-b89f-089b-4b544957d150@gmail.com">Am 10.11.21 um 00:04 schrieb Philip Yang: <blockquote type="cite">IH ring1 is used to process GPU retry fault, overflow is enabled to drain retry fault before unmapping the range, wptr may pass rptr, amdgpu_ih_process should check rptr equals to the latest wptr to exit, otherwise it will continue to recover outdatad retry fault after drain retry fault is done, and generate false GPU vm fault because range is unmapped from cpu. Signed-off-by: Philip Yang <a class="moz-txt-link-rfc2396E" href="mailto:Philip.Yang@amd.com"><Philip.Yang@amd.com></a> --- drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c index f3d62e196901..d1ef61811169 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c @@ -223,7 +223,7 @@ int amdgpu_ih_wait_on_checkpoint_process(struct amdgpu_device *adev, */ int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih) { - unsigned int count = AMDGPU_IH_MAX_NUM_IVS; + unsigned int count; u32 wptr; if (!ih->enabled || adev->shutdown) @@ -232,6 +232,8 @@ int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih) wptr = amdgpu_ih_get_wptr(adev, ih); restart_ih: + count = AMDGPU_IH_MAX_NUM_IVS; + </blockquote> This looks like a bugfix to me and should probably be in a separate patch with CC: stable. </blockquote> ok, will add separate patch for this fix. I think this bug shows up now for ring1 because retry fault is burst and flooding even after filter. <blockquote type="cite" cite="mid:1862b795-3401-b89f-089b-4b544957d150@gmail.com"> <blockquote type="cite"> DRM_DEBUG("%s: rptr %d, wptr %d\n", __func__, ih->rptr, wptr); /* Order reading of wptr vs. reading of IH ring data */ @@ -240,6 +242,9 @@ int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih) while (ih->rptr != wptr && --count) { amdgpu_irq_dispatch(adev, ih); ih->rptr &= ih->ptr_mask; + + if (ih == &adev->irq.ih1) + wptr = amdgpu_ih_get_wptr(adev, ih); </blockquote> Well that handling does not really make much sense. The AMDGPU_IH_MAX_NUM_IVS define controls how many IVs we can process before checking the wptr again. </blockquote> It is hard to understand, this debug log can explain more details, with this debug message patch diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c index ed6f8d24280b..8859f2bb11b1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c @@ -234,10 +235,12 @@ int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih) return IRQ_NONE; wptr = amdgpu_ih_get_wptr(adev, ih); + if (ih == &adev->irq.ih1) + pr_debug("entering rptr 0x%x, wptr 0x%x\n", ih->rptr, wptr); restart_ih: + if (ih == &adev->irq.ih1) + pr_debug("starting rptr 0x%x, wptr 0x%x\n", ih->rptr, wptr); /* Order reading of wptr vs. reading of IH ring data */ rmb(); @@ -245,8 +248,12 @@ int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih) while (ih->rptr != wptr && --count) { amdgpu_irq_dispatch(adev, ih); ih->rptr &= ih->ptr_mask; + if (ih == &adev->irq.ih1) { + pr_debug("rptr 0x%x, old wptr 0x%x, new wptr 0x%x\n", + ih->rptr, wptr, + amdgpu_ih_get_wptr(adev, ih)); + } } amdgpu_ih_set_rptr(adev, ih); @@ -257,6 +264,8 @@ int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih) if (wptr != ih->rptr) goto restart_ih; + if (ih == &adev->irq.ih1) + pr_debug("exiting rptr 0x%x, wptr 0x%x\n", ih->rptr, wptr); return IRQ_HANDLED; } This is log, timing 48.807028, ring1 drain is done, rptr == wptr, ring1 is empty, but the loop continues, to handle outdated retry fault. [ 48.802231] amdgpu_ih_process:243: amdgpu: starting rptr 0x520, wptr 0xd20 [ 48.802235] amdgpu_ih_process:254: amdgpu: rptr 0x540, old wptr 0xd20, new wptr 0xd20 [ 48.802256] amdgpu_ih_process:254: amdgpu: rptr 0x560, old wptr 0xd20, new wptr 0xd20 [ 48.802260] amdgpu_ih_process:254: amdgpu: rptr 0x580, old wptr 0xd20, new wptr 0xd20 [ 48.802281] amdgpu_ih_process:254: amdgpu: rptr 0x5a0, old wptr 0xd20, new wptr 0xd20 [ 48.802314] amdgpu_ih_process:254: amdgpu: rptr 0x5c0, old wptr 0xd20, new wptr 0xce0 [ 48.802335] amdgpu_ih_process:254: amdgpu: rptr 0x5e0, old wptr 0xd20, new wptr 0xce0 [ 48.802356] amdgpu_ih_process:254: amdgpu: rptr 0x600, old wptr 0xd20, new wptr 0xce0 [ 48.802376] amdgpu_ih_process:254: amdgpu: rptr 0x620, old wptr 0xd20, new wptr 0xce0 [ 48.802396] amdgpu_ih_process:254: amdgpu: rptr 0x640, old wptr 0xd20, new wptr 0xce0 [ 48.802401] amdgpu_ih_process:254: amdgpu: rptr 0x660, old wptr 0xd20, new wptr 0xce0 [ 48.802421] amdgpu_ih_process:254: amdgpu: rptr 0x680, old wptr 0xd20, new wptr 0xce0 [ 48.802442] amdgpu_ih_process:254: amdgpu: rptr 0x6a0, old wptr 0xd20, new wptr 0xce0 [ 48.802463] amdgpu_ih_process:254: amdgpu: rptr 0x6c0, old wptr 0xd20, new wptr 0xce0 [ 48.802483] amdgpu_ih_process:254: amdgpu: rptr 0x6e0, old wptr 0xd20, new wptr 0xce0 [ 48.802503] amdgpu_ih_process:254: amdgpu: rptr 0x700, old wptr 0xd20, new wptr 0xce0 [ 48.802523] amdgpu_ih_process:254: amdgpu: rptr 0x720, old wptr 0xd20, new wptr 0xce0 [ 48.802544] amdgpu_ih_process:254: amdgpu: rptr 0x740, old wptr 0xd20, new wptr 0xce0 [ 48.802565] amdgpu_ih_process:254: amdgpu: rptr 0x760, old wptr 0xd20, new wptr 0xce0 [ 48.802569] amdgpu_ih_process:254: amdgpu: rptr 0x780, old wptr 0xd20, new wptr 0xce0 [ 48.804392] amdgpu_ih_process:254: amdgpu: rptr 0x7a0, old wptr 0xd20, new wptr 0xf00 [ 48.806122] amdgpu_ih_process:254: amdgpu: rptr 0x7c0, old wptr 0xd20, new wptr 0x840 [ 48.806155] amdgpu_ih_process:254: amdgpu: rptr 0x7e0, old wptr 0xd20, new wptr 0x840 [ 48.806965] amdgpu_ih_process:254: amdgpu: rptr 0x800, old wptr 0xd20, new wptr 0x840 [ 48.806995] amdgpu_ih_process:254: amdgpu: rptr 0x820, old wptr 0xd20, new wptr 0x840 [ 48.807028] amdgpu_ih_process:254: amdgpu: rptr 0x840, old wptr 0xd20, new wptr 0x840 [ 48.807063] amdgpu_ih_process:254: amdgpu: rptr 0x860, old wptr 0xd20, new wptr 0x840 [ 48.808421] amdgpu_ih_process:254: amdgpu: rptr 0x880, old wptr 0xd20, new wptr 0x840 Cause this gpu vm fault dump because address is unmapped from cpu. [ 48.807071] svm_range_restore_pages:2617: amdgpu: restoring svms 0x00000000733bf007 fault address 0x7f8a6991f [ 48.807170] svm_range_restore_pages:2631: amdgpu: failed to find prange svms 0x00000000733bf007 address [0x7f8a6991f] [ 48.807179] svm_range_get_range_boundaries:2348: amdgpu: VMA does not exist in address [0x7f8a6991f] [ 48.807185] svm_range_restore_pages:2635: amdgpu: failed to create unregistered range svms 0x00000000733bf007 address [0x7f8a6991f] [ 48.807929] amdgpu 0000:25:00.0: amdgpu: [mmhub0] retry page fault (src_id:0 ring:0 vmid:8 pasid:32770, for process kfdtest pid 3969 thread kfdtest pid 3969) [ 48.808219] amdgpu 0000:25:00.0: amdgpu: in page starting at address 0x00007f8a6991f000 from IH client 0x12 (VMC) [ 48.808230] amdgpu 0000:25:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00800031 <blockquote type="cite" cite="mid:1862b795-3401-b89f-089b-4b544957d150@gmail.com">We could of course parameterize that so that we check the wptr after each IV on IH1, but please not hard coded like this. Regards, Christian. <blockquote type="cite"> } amdgpu_ih_set_rptr(adev, ih); </blockquote> </blockquote> </body> </html>