[PATCH 2/2] drm/amdgpu: Let KFD use more VMIDs on Arcturus

[Felix Kuehling]

Am 2020-06-25 um 11:50 a.m. schrieb Christian König:
> Am 25.06.20 um 17:43 schrieb Felix Kuehling:
>> Am 2020-06-25 um 11:38 a.m. schrieb Christian König:
>>> Am 25.06.20 um 17:15 schrieb Felix Kuehling:
>>>> Am 2020-06-25 um 4:19 a.m. schrieb Christian König:
>>>>> Am 25.06.20 um 05:18 schrieb Felix Kuehling:
>>>>>> When there is no graphics support, KFD can use more of the VMIDs.
>>>>>> Graphics
>>>>>> VMIDs are only used for video decoding/encoding and post processing.
>>>>>> With
>>>>>> two VCE engines, there is no reason to reserve more than 2 VMIDs for
>>>>>> that.
>>>>> IIRC the expectation is that we still use the compute queues for post
>>>>> processing and not the KFD.
>>>>>
>>>>> So we will need at least VMIDs for that as well.
>>>> Correct. Post processing uses compute queues and VMIDs in the GFXHUB.
>>>> VCE uses VMIDs in the MMHUB. I believe in Mesa they use the same VM
>>>> context. So can't they share the same VMIDs?
>>> Ah! Good point, But we still need at least 3 VMID when VMID
>>> reservation is active.
>> I don't know anything about that VMID reservation feature. What is it
>> used for? Who is using it? How many VMIDs can be reserved?
>>
>> If one VMID is reserved, there would still be one VMID left for video
>> post processing. That's not ideal, but I don't think it would be fatal.
>> But is it a realistic use case that VMID reservation and ROCm+video
>> processing would happen on the same system at the same time?
>
> VMID reservation is used for debugging and yes there can only be one
> reserved.
>
> But I think we need at least two for dynamic assignment or we might
> run into a BUG_ON() while giving VMIDs to jobs.

I don't see any BUGs or BUG_ONs in amdgpu_ids.c. What should I be
looking out for?


> But I certainly need to test this as well. It's possible that 1 VMID
> indeed works as expected.

I could run the test, if you describe the problematic scenario you have
in mind.

Thanks,
  Felix


>
> Regards,
> Christian.
>
>>
>> Thanks,
>>    Felix
>>
>>
>>> I don't think you can go below this.
>>>
>>> Regards,
>>> Christian.
>>>
>>>> Regards,
>>>>     Felix
>>>>
>>>>
>>>>>> Signed-off-by: Felix Kuehling <Felix.Kuehling at amd.com>
>>>>>> ---
>>>>>>     drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 11 ++++++++---
>>>>>>     1 file changed, 8 insertions(+), 3 deletions(-)
>>>>>>
>>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
>>>>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
>>>>>> index 6e10b42c57e5..3470929e5b8e 100644
>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
>>>>>> @@ -1245,10 +1245,15 @@ static int gmc_v9_0_sw_init(void *handle)
>>>>>>         /*
>>>>>>          * number of VMs
>>>>>>          * VMID 0 is reserved for System
>>>>>> -     * amdgpu graphics/compute will use VMIDs 1-7
>>>>>> -     * amdkfd will use VMIDs 8-15
>>>>>> +     * amdgpu graphics/compute will use VMIDs 1..n-1
>>>>>> +     * amdkfd will use VMIDs n..15
>>>>>> +     *
>>>>>> +     * The first KFD VMID is 8 for GPUs with graphics, 3 for
>>>>>> +     * compute-only GPUs. On compute-only GPUs that leaves 2 VMIDs
>>>>>> +     * for video processing.
>>>>>>          */
>>>>>> -    adev->vm_manager.first_kfd_vmid = 8;
>>>>>> +    adev->vm_manager.first_kfd_vmid =
>>>>>> +        adev->asic_type == CHIP_ARCTURUS ? 3 : 8;
>>>>>>           amdgpu_vm_manager_init(adev);
>>>>>>     
>