[PATCH 1/2] drm/amdkfd: set prefetch GPU has access attribute

[Felix Kuehling]

Am 2021-05-05 um 4:01 p.m. schrieb Felix Kuehling:
> Am 2021-05-05 um 1:56 p.m. schrieb Philip Yang:
>> After svm range prefetch and migrate to GPU, if GPU has no access or
>> access in place attribute, add GPU to range access_bitmap to be able to
>> update GPU page table.
> I don't think we should change the bitmap_access, because this is
> persistent and can be queried by user mode. Changing this would not be
> expected by the application. I think instead we should either pass the
> prefetch location to the gpuidx parameter of validate_and_map (we should
> only do that is xnack is enabled), or change the logic in
> validate_and_map to add the prefetch location to the bitmap.

Actually, I don't think we need any special rules here. If the memory is
not accessible by the prefetch GPU, the memory should not get mapped. It
doesn't sound like a very useful thing to do, but that's how the API is
defined.

Memory should get mapped to the local GPU (if it has access) and to any
other GPUs that have access-in-place.

If you're working around a problem of a range not being accessible after
it was created in a page fault, then this needs to be fixed elsewhere.
With XNACK enabled, all GPUs should have access to all memory by
default. If the bitmap_access is not set on a new range created in a
page fault, this is a bug in how we created that range.

Regards,
  Felix


>
> I think to make this reliable we'll need to add more sanity checks to
> prefetch to make sure the prefetch location is compatible with all GPUs
> that have access_in_place. Otherwise we have to reject the prefetch.
>
> Regards,
>   Felix
>
>
>> Signed-off-by: Philip Yang <Philip.Yang at amd.com>
>> ---
>>  drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 11 +++++++++++
>>  1 file changed, 11 insertions(+)
>>
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
>> index d9111fea724b..3b640772d9c5 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
>> @@ -2869,6 +2869,17 @@ svm_range_set_attr(struct kfd_process *p, uint64_t start, uint64_t size,
>>  			continue;
>>  		}
>>  
>> +		if (migrated && prange->prefetch_loc) {
>> +			int gpuidx;
>> +
>> +			gpuidx = kfd_process_gpuidx_from_gpuid(p,
>> +							prange->prefetch_loc);
>> +			if (gpuidx >= 0 &&
>> +			    !test_bit(gpuidx, prange->bitmap_access) &&
>> +			    !test_bit(gpuidx, prange->bitmap_aip))
>> +				bitmap_set(prange->bitmap_access, gpuidx, 1);
>> +		}
>> +
>>  		r = svm_range_validate_and_map(mm, prange, MAX_GPU_INSTANCE,
>>  					       true, true);
>>  		if (r)
> _______________________________________________
> amd-gfx mailing list
> amd-gfx at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx