[PATCH] drm/amdkfd: Fix partial migrate issue

[Felix Kuehling]

On 2025-01-05 20:45, Deng, Emily wrote:
> [AMD Official Use Only - AMD Internal Distribution Only]
>
>> -----Original Message-----
>> From: Kuehling, Felix <Felix.Kuehling at amd.com>
>> Sent: Saturday, January 4, 2025 7:10 AM
>> To: Deng, Emily <Emily.Deng at amd.com>; amd-gfx at lists.freedesktop.org
>> Subject: Re: [PATCH] drm/amdkfd: Fix partial migrate issue
>>
>>
>> On 2025-01-02 19:06, Emily Deng wrote:
>>> For partial migrate from ram to vram, the migrate->cpages is not equal
>>> to migrate->npages, should use migrate->npages to check all needed
>>> migrate pages which could be copied or not.
>>>
>>> And only need to set those pages could be migrated to migrate->dst[i],
>>> or the migrate_vma_pages will migrate the wrong pages based on the migrate-
>>> dst[i].
>>>
>>> Signed-off-by: Emily Deng <Emily.Deng at amd.com>
>> Good catch. But I think it's still not quite right. See inline.
>>
>>
>>> ---
>>>    drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 8 ++++----
>>>    1 file changed, 4 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c
>>> b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c
>>> index 4b275937d05e..5c96c2d425e3 100644
>>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c
>>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c
>>> @@ -278,7 +278,7 @@ svm_migrate_copy_to_vram(struct kfd_node *node, struct
>> svm_range *prange,
>>>                        struct migrate_vma *migrate, struct dma_fence **mfence,
>>>                        dma_addr_t *scratch, uint64_t ttm_res_offset)
>>>    {
>>> -    uint64_t npages = migrate->cpages;
>>> +    uint64_t npages = migrate->npages;
>>>       struct amdgpu_device *adev = node->adev;
>>>       struct device *dev = adev->dev;
>>>       struct amdgpu_res_cursor cursor;
>>> @@ -299,9 +299,6 @@ svm_migrate_copy_to_vram(struct kfd_node *node, struct
>> svm_range *prange,
>>>               struct page *spage;
>>>
>>>               dst[i] = cursor.start + (j << PAGE_SHIFT);
>>> -            migrate->dst[i] = svm_migrate_addr_to_pfn(adev, dst[i]);
>>> -            svm_migrate_get_vram_page(prange, migrate->dst[i]);
>>> -            migrate->dst[i] = migrate_pfn(migrate->dst[i]);
>>>
>>>               spage = migrate_pfn_to_page(migrate->src[i]);
>>>               if (spage && !is_zone_device_page(spage)) {
>> We should also check (migrate->src[i] & MIGRATE_PFN_MIGRATE) to catch only the
>> pages that are ready to migrate (i.e. not already in device memory and not pinned by
>> someone else).
>>
>> Regards,
>>    Felix
>> The code " if (spage && !is_zone_device_page(spage)) {" is already checked whether the page could be migrated or not. Because after called migrate_vma_setup, for those pages couldn't be migrated it will be set to NULL.

The page may be non-NULL. If the MIGRATE_PFN_MIGRATE is not set, the 
page could not be collected for migration by migrate_vma_setup and it 
should not be migrated. See 
https://elixir.bootlin.com/linux/v6.12.6/source/mm/migrate_device.c#L470

Regards,
   Felix


> Emily Deng
> Best Wishes
>
>
>>> @@ -345,6 +342,9 @@ svm_migrate_copy_to_vram(struct kfd_node *node, struct
>> svm_range *prange,
>>>               } else {
>>>                       j++;
>>>               }
>>> +            migrate->dst[i] = svm_migrate_addr_to_pfn(adev, dst[i]);
>>> +            svm_migrate_get_vram_page(prange, migrate->dst[i]);
>>> +            migrate->dst[i] = migrate_pfn(migrate->dst[i]);
>>>       }
>>>
>>>       r = svm_migrate_copy_memory_gart(adev, src + i - j, dst + i - j, j,