[PATCH v4] drm/amdgpu: Restore msix after FLR

[Lazar, Lijo]

On 7/2/2021 8:53 AM, Peng Ju Zhou wrote:
> From: "Emily.Deng" <Emily.Deng at amd.com>
> 
> After FLR, the msix will be cleared, so need to re-enable it.
> 
> Signed-off-by: Emily.Deng <Emily.Deng at amd.com>
> Signed-off-by: Peng Ju Zhou <PengJu.Zhou at amd.com>
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 14 ++++++++++++++
>   1 file changed, 14 insertions(+)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> index 90f50561b43a..034420c38352 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> @@ -277,6 +277,19 @@ static bool amdgpu_msi_ok(struct amdgpu_device *adev)
>   	return true;
>   }
>   
> +void amdgpu_restore_msix(struct amdgpu_device *adev)
> +{
> +	u16 ctrl;
> +
> +	pci_read_config_word(adev->pdev, adev->pdev->msix_cap + PCI_MSIX_FLAGS, &ctrl);
> +	if (!(ctrl & PCI_MSIX_FLAGS_ENABLE))
> +		return;
> +
> +	ctrl &= ~PCI_MSIX_FLAGS_ENABLE;
> +	pci_write_config_word(adev->pdev, adev->pdev->msix_cap + PCI_MSIX_FLAGS, ctrl);
> +	ctrl |= PCI_MSIX_FLAGS_ENABLE;
> +	pci_write_config_word(adev->pdev, adev->pdev->msix_cap + PCI_MSIX_FLAGS, ctrl);
This field behavior is determined by PCIE spec and toggling is not 
required to enable MSIX. Also, FLR will clear this field on PF, so the 
ENABLE flag check to see if MSIX is already enabled doesn't make sense 
for PF case.

 From the code logic, a rough guess is this code is trying to reset the 
VF's config space field in PF after VF FLR and enable MSI-X back. If 
that is the case, make an explicit check so that this is only done on VF 
devices.

Thanks,
Lijo

> +}
>   /**
>    * amdgpu_irq_init - initialize interrupt handling
>    *
> @@ -558,6 +571,7 @@ void amdgpu_irq_gpu_reset_resume_helper(struct amdgpu_device *adev)
>   {
>   	int i, j, k;
>   
> +	amdgpu_restore_msix(adev);
>   	for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) {
>   		if (!adev->irq.client[i].sources)
>   			continue;
>