[PATCH 2/4] drm/amdgpu: allow higher level PD invalidations

Tom St Denis tom.stdenis at amd.com
Mon Jan 27 17:37:13 UTC 2020


The offending PDE address is: "3ffffeff600000"

Which looks like it was sign extended into the "reserved" section 
between bits 40:58 (0x3fff) hence triggering the BUG() in the gmc_v8 driver.

Tom

On 2020-01-27 9:57 a.m., Tom St Denis wrote:
> Reverting this patch avoids the gmc_v8 errors (I previously sent 
> kernel logs, here's one for convenience):
>
> [  358.554335] ------------[ cut here ]------------
> [  358.554338] kernel BUG at drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c:725!
> [  358.554351] invalid opcode: 0000 [#1] SMP NOPTI
> [  358.554354] CPU: 0 PID: 4551 Comm: Xwayland Not tainted 5.4.0-rc7+ #14
> [  358.554355] Hardware name: System manufacturer System Product 
> Name/TUF B350M-PLUS GAMING, BIOS 5220 09/12/2019
> [  358.554452] RIP: 0010:gmc_v8_0_get_vm_pde+0x10/0x20 [amdgpu]
> [  358.554455] Code: 31 f6 48 89 df e8 30 e9 ff ff e9 28 ff ff ff e8 
> 16 d6 21 f9 66 0f 1f 44 00 00 48 b8 ff 0f 00 00 00 ff ff ff 48 85 02 
> 75 01 c3 <0f> 0b 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 55 48 89 fd 
> e8 c7
> [  358.554456] RSP: 0018:ffffa28142287a00 EFLAGS: 00010206
> [  358.554458] RAX: ffffff0000000fff RBX: 0000000000000000 RCX: 
> ffffa28142287a78
> [  358.554459] RDX: ffffa28142287a50 RSI: 0000000000000002 RDI: 
> ffff8b9be15e0000
> [  358.554460] RBP: 0000000000000001 R08: 0000000000000000 R09: 
> 0000000000000000
> [  358.554461] R10: 000000000000000f R11: 0000000000000406 R12: 
> 0000000000002030
> [  358.554462] R13: 003ffffefea00000 R14: 0000000000101c00 R15: 
> ffffa28142287af0
> [  358.554464] FS:  00007f180a48ba80(0000) GS:ffff8b9be6c00000(0000) 
> knlGS:0000000000000000
> [  358.554465] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [  358.554466] CR2: 00007f3de8f5dcc0 CR3: 00000002170c8000 CR4: 
> 00000000001406f0
> [  358.554467] Call Trace:
> [  358.554502]  amdgpu_vm_update_ptes+0x28a/0x7f0 [amdgpu]
> [  358.554534]  ? amdgpu_sync_resv+0x34/0x190 [amdgpu]
> [  358.554565]  amdgpu_vm_bo_update_mapping+0x12b/0x160 [amdgpu]
> [  358.554596]  amdgpu_vm_bo_update+0x333/0x6a0 [amdgpu]
> [  358.554626]  amdgpu_gem_va_ioctl+0x3c1/0x3e0 [amdgpu]
> [  358.554658]  ? amdgpu_gem_va_map_flags+0x60/0x60 [amdgpu]
> [  358.554663]  drm_ioctl_kernel+0xa5/0xf0
> [  358.554665]  drm_ioctl+0x1df/0x366
> [  358.554695]  ? amdgpu_gem_va_map_flags+0x60/0x60 [amdgpu]
> [  358.554698]  ? __switch_to_asm+0x34/0x70
> [  358.554699]  ? __switch_to_asm+0x40/0x70
> [  358.554701]  ? __switch_to_asm+0x34/0x70
> [  358.554702]  ? __switch_to_asm+0x40/0x70
> [  358.554703]  ? __switch_to_asm+0x34/0x70
> [  358.554704]  ? __switch_to_asm+0x40/0x70
> [  358.554731]  amdgpu_drm_ioctl+0x44/0x80 [amdgpu]
> [  358.554735]  do_vfs_ioctl+0x3f0/0x650
> [  358.554737]  ? __schedule+0x28c/0x5a0
> [  358.554738]  ksys_ioctl+0x59/0x90
> [  358.554740]  __x64_sys_ioctl+0x11/0x20
> [  358.554743]  do_syscall_64+0x43/0x110
> [  358.554745]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
> [  358.554747] RIP: 0033:0x7f1809e6638b
> [  358.554749] Code: 0f 1e fa 48 8b 05 fd 9a 0c 00 64 c7 00 26 00 00 
> 00 48 c7 c0 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa b8 10 00 00 
> 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d cd 9a 0c 00 f7 d8 64 89 
> 01 48
> [  358.554750] RSP: 002b:00007fffac20a638 EFLAGS: 00000246 ORIG_RAX: 
> 0000000000000010
> [  358.554751] RAX: ffffffffffffffda RBX: 00007fffac20a690 RCX: 
> 00007f1809e6638b
> [  358.554752] RDX: 00007fffac20a690 RSI: 00000000c0286448 RDI: 
> 000000000000000e
> [  358.554753] RBP: 00000000c0286448 R08: 0000000101600000 R09: 
> 000000000000000e
> [  358.554754] R10: 00000000000000e0 R11: 0000000000000246 R12: 
> 0000000000000000
> [  358.554754] R13: 000000000000000e R14: 0000000000000001 R15: 
> 0000563d48bfd1f0
> [  358.554756] Modules linked in: amdgpu gpu_sched ttm r8152 efivarfs
> [  358.554790] ---[ end trace e0d54f6c49902356 ]---
> [  358.554824] RIP: 0010:gmc_v8_0_get_vm_pde+0x10/0x20 [amdgpu]
>
> (the gmc_v8 bug triggers regardless of whether I'm running piglit on 
> my headless vega20 or directly on the carrizo).
>
> However, with patch 2 of 4 reverted I then get:
>
> [ 1471.338089] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.338647] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.339807] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.341699] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.342348] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.342474] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.342532] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.342583] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.342636] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.342694] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.342745] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.342796] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.343555] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.350270] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.350351] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.350395] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.351895] amdgpu 0000:00:01.0: failed to clear page tables on GEM 
> object close (-2)
> [ 1471.353995] amdgpu 0000:00:01.0: failed to clear page tables on GEM 
> object close (-2)
> [ 1471.354179] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.354190] amdgpu 0000:00:01.0: failed to clear page tables on GEM 
> object close (-2)
> [ 1471.354252] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.354259] amdgpu 0000:00:01.0: failed to clear page tables on GEM 
> object close (-2)
> [ 1471.354302] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.354308] amdgpu 0000:00:01.0: failed to clear page tables on GEM 
> object close (-2)
> [ 1471.354351] [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't 
> update BO_VA (-2)
> [ 1471.354356] amdgpu 0000:00:01.0: failed to clear page tables on GEM 
> object close (-2)
>
> So clearly that's not the fix either :-/
>
> (all on top of the latest drm-next from this morning).
>
> Tom
>
>
> On 2020-01-23 9:21 a.m., Christian König wrote:
>> Allow partial invalidation on unallocated PDs. This is useful when we
>> need to silence faults to stop interrupt floods on Vega.
>>
>> Signed-off-by: Christian König <christian.koenig at amd.com>
>> ---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 23 ++++++++++++++++++-----
>>   1 file changed, 18 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>> index 8119f32ca94d..0f79c17118bf 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>> @@ -1467,9 +1467,8 @@ static int amdgpu_vm_update_ptes(struct 
>> amdgpu_vm_update_params *params,
>>                * smaller than the address shift. Go to the next
>>                * child entry and try again.
>>                */
>> -            if (!amdgpu_vm_pt_descendant(adev, &cursor))
>> -                return -ENOENT;
>> -            continue;
>> +            if (amdgpu_vm_pt_descendant(adev, &cursor))
>> +                continue;
>>           } else if (frag >= parent_shift) {
>>               /* If the fragment size is even larger than the parent
>>                * shift we should go up one level and check it again.
>> @@ -1480,8 +1479,19 @@ static int amdgpu_vm_update_ptes(struct 
>> amdgpu_vm_update_params *params,
>>           }
>>             pt = cursor.entry->base.bo;
>> -        if (!pt)
>> -            return -ENOENT;
>> +        if (!pt) {
>> +            /* We need all PDs and PTs for mapping something, */
>> +            if (flags & AMDGPU_PTE_VALID)
>> +                return -ENOENT;
>> +
>> +            /* but unmapping something can happen at a higher
>> +             * level. */
>> +            if (!amdgpu_vm_pt_ancestor(&cursor))
>> +                return -EINVAL;
>> +
>> +            pt = cursor.entry->base.bo;
>> +            shift = parent_shift;
>> +        }
>>             /* Looks good so far, calculate parameters for the update */
>>           incr = (uint64_t)AMDGPU_GPU_PAGE_SIZE << shift;
>> @@ -1495,6 +1505,9 @@ static int amdgpu_vm_update_ptes(struct 
>> amdgpu_vm_update_params *params,
>>               uint64_t upd_end = min(entry_end, frag_end);
>>               unsigned nptes = (upd_end - frag_start) >> shift;
>>   +            /* This can happen when we set higher level PDs to
>> +             * silent to stop fault floods. */
>> +            nptes = max(nptes, 1u);
>>               amdgpu_vm_update_flags(params, pt, cursor.level,
>>                              pe_start, dst, nptes, incr,
>>                              flags | AMDGPU_PTE_FRAG(frag));


More information about the amd-gfx mailing list