[PATCH 2/2] drm/amdgpu: Use offsets local to VCN in VF

[Lazar, Lijo]

On 3/5/2024 2:48 PM, Christian König wrote:
> Am 05.03.24 um 10:03 schrieb Lazar, Lijo:
>>
>> On 3/5/2024 2:24 PM, Christian König wrote:
>>>
>>> Am 05.03.24 um 07:40 schrieb Lijo Lazar:
>>>> For VCN 4.0.3, use only the local addressing scheme while in VF
>>>> mode. This includes addressing scheme used for HUB offsets.
>>>>
>>>> Signed-off-by: Lijo Lazar <lijo.lazar at amd.com>
>>>> ---
>>>>    drivers/gpu/drm/amd/amdgpu/vcn_v4_0_3.c | 20 +++++++++++++++++++-
>>>>    1 file changed, 19 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0_3.c
>>>> b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0_3.c
>>>> index 7b5ad13b618e..a27f3f260aab 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0_3.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0_3.c
>>>> @@ -1381,6 +1381,24 @@ static uint64_t
>>>> vcn_v4_0_3_unified_ring_get_wptr(struct amdgpu_ring *ring)
>>>>                        regUVD_RB_WPTR);
>>>>    }
>>>>    +static void vcn_v4_0_3_enc_ring_emit_vm_flush(struct amdgpu_ring
>>>> *ring,
>>>> +                unsigned int vmid, uint64_t pd_addr)
>>>> +{
>>>> +    struct amdgpu_vmhub *hub;
>>>> +
>>>> +    /* For VF, only local offsets should be used */
>>>> +    if (amdgpu_sriov_vf(ring->adev))
>>>> +        ring->vm_hub = AMDGPU_MMHUB0(0);
>>> That is clearly a no-go since the vm_hub must be statically and can't be
>>> changed here.
>>>
>> After HUB allocation, the only usage of this hub pointer is to calculate
>> use the right offset. We still want VCN to use the right hub, only thing
>> is register offsets in MMHUB(0) are equal to 'local offsets'.
> 
> The vm_hub is a static setup describing how the engine works. You
> basically just insert an illegal value here to fix your register offset
> calculation.
> 
> That is absolutely *not* something you can do.
> 
> What exactly is the requirement?
> 

The requirement is this way -

We have multiple MMHUBs and each VCN can talk only to the local MMHUB.

The absolute register offsets we store for each for an example reg at
offset x is
	HUB0 = x, HUB1 = x + Y (stride), HUB2 = x + 2Y and so forth.

However VCN cannot use the absolute register offset in the packet in VF
mode, instead it should use a local offset for the local HUB in VF mode.
A local offset of a register in the HUB is exactly the same as offset of
HUB0 = x.

What we do here is assign the HUB as regular here
	ring->vm_hub = AMDGPU_MMHUB0(adev->vcn.inst[i].aid_id);

so that amdgpu_gmc_allocate_vm_inv_eng is allocated in the right way.

Then at the time of flush, switch it back to point to HUB0 so that it
uses the offsets of HUB0 in the packet. Actually, it's not required to
switch this every time, one-time is fine after
amdgpu_gmc_allocate_vm_inv_eng() is complete.

Thanks,
Lijo

> Regards,
> Christian.
> 
>>
>> Thanks,
>> Lijo
>>
>>> Regards,
>>> Christian.
>>>
>>>> +    hub = &ring->adev->vmhub[ring->vm_hub];
>>>> +
>>>> +    pd_addr = amdgpu_gmc_emit_flush_gpu_tlb(ring, vmid, pd_addr);
>>>> +
>>>> +    /* wait for reg writes */
>>>> +    vcn_v2_0_enc_ring_emit_reg_wait(ring, hub->ctx0_ptb_addr_lo32 +
>>>> +                    vmid * hub->ctx_addr_distance,
>>>> +                    lower_32_bits(pd_addr), 0xffffffff);
>>>> +}
>>>> +
>>>>    static void vcn_v4_0_3_ring_emit_hdp_flush(struct amdgpu_ring *ring)
>>>>    {
>>>>        /* VCN engine access for HDP flush doesn't work when RRMT is
>>>> enabled.
>>>> @@ -1443,7 +1461,7 @@ static const struct amdgpu_ring_funcs
>>>> vcn_v4_0_3_unified_ring_vm_funcs = {
>>>>        .emit_ib_size = 5, /* vcn_v2_0_enc_ring_emit_ib */
>>>>        .emit_ib = vcn_v2_0_enc_ring_emit_ib,
>>>>        .emit_fence = vcn_v2_0_enc_ring_emit_fence,
>>>> -    .emit_vm_flush = vcn_v2_0_enc_ring_emit_vm_flush,
>>>> +    .emit_vm_flush = vcn_v4_0_3_enc_ring_emit_vm_flush,
>>>>        .emit_hdp_flush = vcn_v4_0_3_ring_emit_hdp_flush,
>>>>        .test_ring = amdgpu_vcn_enc_ring_test_ring,
>>>>        .test_ib = amdgpu_vcn_unified_ring_test_ib,
>