[PATCH 1/2] drm/amdkfd: introduce dummy cache info for property asic

[Liang, Prike]

[Public]

-----Original Message-----
From: Kuehling, Felix <Felix.Kuehling at amd.com>
Sent: Friday, October 21, 2022 1:11 PM
To: Liang, Prike <Prike.Liang at amd.com>; amd-gfx at lists.freedesktop.org
Cc: Deucher, Alexander <Alexander.Deucher at amd.com>; Zhang, Yifan <Yifan1.Zhang at amd.com>; Huang, Ray <Ray.Huang at amd.com>; Liu, Aaron <Aaron.Liu at amd.com>
Subject: Re: [PATCH 1/2] drm/amdkfd: introduce dummy cache info for property asic

Am 2022-10-20 um 21:50 schrieb Liang, Prike:
> [Public]
>
> -----Original Message-----
> From: Kuehling, Felix <Felix.Kuehling at amd.com>
> Sent: Friday, October 21, 2022 12:03 AM
> To: Liang, Prike <Prike.Liang at amd.com>; amd-gfx at lists.freedesktop.org
> Cc: Deucher, Alexander <Alexander.Deucher at amd.com>; Zhang, Yifan
> <Yifan1.Zhang at amd.com>; Huang, Ray <Ray.Huang at amd.com>; Liu, Aaron
> <Aaron.Liu at amd.com>
> Subject: Re: [PATCH 1/2] drm/amdkfd: introduce dummy cache info for
> property asic
>
>
> Am 2022-10-20 um 05:15 schrieb Prike Liang:
>> This dummy cache info will enable kfd base function support.
>>
>> Signed-off-by: Prike Liang <Prike.Liang at amd.com>
>> ---
>>    drivers/gpu/drm/amd/amdkfd/kfd_crat.c | 55 +++++++++++++++++++++++++--
>>    1 file changed, 52 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
>> b/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
>> index cd5f8b219bf9..960046e43b7a 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
>> @@ -795,6 +795,54 @@ static struct kfd_gpu_cache_info yellow_carp_cache_info[] = {
>>        },
>>    };
>>
>> +static struct kfd_gpu_cache_info dummy_cache_info[] = {
>> +     {
>> +             /* TCP L1 Cache per CU */
>> +             .cache_size = 16,
>> +             .cache_level = 1,
>> +             .flags = (CRAT_CACHE_FLAGS_ENABLED |
>> +                             CRAT_CACHE_FLAGS_DATA_CACHE |
>> +                             CRAT_CACHE_FLAGS_SIMD_CACHE),
>> +             .num_cu_shared = 1,
>> +     },
>> +     {
>> +             /* Scalar L1 Instruction Cache per SQC */
>> +             .cache_size = 32,
>> +             .cache_level = 1,
>> +             .flags = (CRAT_CACHE_FLAGS_ENABLED |
>> +                             CRAT_CACHE_FLAGS_INST_CACHE |
>> +                             CRAT_CACHE_FLAGS_SIMD_CACHE),
>> +             .num_cu_shared = 2,
>> +     },
>> +     {
>> +             /* Scalar L1 Data Cache per SQC */
>> +             .cache_size = 16,
>> +             .cache_level = 1,
>> +             .flags = (CRAT_CACHE_FLAGS_ENABLED |
>> +                             CRAT_CACHE_FLAGS_DATA_CACHE |
>> +                             CRAT_CACHE_FLAGS_SIMD_CACHE),
>> +             .num_cu_shared = 2,
>> +     },
>> +     {
>> +             /* GL1 Data Cache per SA */
>> +             .cache_size = 128,
>> +             .cache_level = 1,
>> +             .flags = (CRAT_CACHE_FLAGS_ENABLED |
>> +                             CRAT_CACHE_FLAGS_DATA_CACHE |
>> +                             CRAT_CACHE_FLAGS_SIMD_CACHE),
>> +             .num_cu_shared = 6,
>> +     },
>> +     {
>> +             /* L2 Data Cache per GPU (Total Tex Cache) */
>> +             .cache_size = 2048,
>> +             .cache_level = 2,
>> +             .flags = (CRAT_CACHE_FLAGS_ENABLED |
>> +                             CRAT_CACHE_FLAGS_DATA_CACHE |
>> +                             CRAT_CACHE_FLAGS_SIMD_CACHE),
>> +             .num_cu_shared = 6,
>> +     },
>> +};
>> +
>>    static void kfd_populated_cu_info_cpu(struct kfd_topology_device *dev,
>>                struct crat_subtype_computeunit *cu)
>>    {
>> @@ -1514,8 +1562,6 @@ static int kfd_fill_gpu_cache_info(struct kfd_dev *kdev,
>>                        num_of_cache_types = ARRAY_SIZE(beige_goby_cache_info);
>>                        break;
>>                case IP_VERSION(10, 3, 3):
>> -             case IP_VERSION(10, 3, 6): /* TODO: Double check these on production silicon */
>> -             case IP_VERSION(10, 3, 7): /* TODO: Double check these on production silicon */
>>                        pcache_info = yellow_carp_cache_info;
>>                        num_of_cache_types = ARRAY_SIZE(yellow_carp_cache_info);
>>                        break;
>> @@ -1528,7 +1574,10 @@ static int kfd_fill_gpu_cache_info(struct kfd_dev *kdev,
>>                                kfd_fill_gpu_cache_info_from_gfx_config(kdev, pcache_info);
>>                        break;
>>                default:
>> -                     return -EINVAL;
>> +                     pcache_info = dummy_cache_info;
>> +                     num_of_cache_types = ARRAY_SIZE(dummy_cache_info);
>> +                     pr_warn("dummy cache info is used temporarily and real cache info need update later.\n");
>> +                     break;
> Could we make this return an error if the amdgpu.exp_hw_support module parameter is not set?
>
> Regards,
>     Felix
>
> [Prike] It's fine to protect this dummy info by checking the parameter amdgpu_exp_hw_support, but it may not friendly to end user by adding the parameter and some guys will still report KFD not enabled for this parameter setting problem. The original idea is the end user will not aware the dummy cache info and only alert the warning message to developer.

I thought the intention was to simplify bring-up but make sure that valid cache info is available by the time a chip goes into production.
Therefore, normal end users should never need to set the amdgpu_exp_hw_support option. I think you're saying that we would go to production with dummy info. That seems like a bad idea to me.

Regards,
   Felix

[Prike]  Sorry for the confusion. In fact, this dummy cache info only used before production and meanwhile needn't require any parameter setting for CQE do KFD test. Anyway if you still have concern on this solution I will add the condition of checking amdgpu_exp_hw_support.

>
>>                }
>>        }
>>