[PATCH 154/207] drm/amdgpu/sriov : Use kiq to do tlb invalidation for gfx10 on sriov

[Alex Deucher]

From: shaoyunl <shaoyun.liu at amd.com>

On SRIOV run time, driver shouldn't directly access invalidation registers through MMIO.
Use kiq to submit wait_reg_mem package for the invalidation

Signed-off-by: shaoyunl <shaoyun.liu at amd.com>
Reviewed-by: Christian König <christian.koenig at amd.com>
Signed-off-by: Alex Deucher <alexander.deucher at amd.com>
---
 drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c |  3 +++
 drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c | 18 ++++++++++++++++++
 2 files changed, 21 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c
index 8935b9b81d68..f0955b325798 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c
@@ -3480,6 +3480,9 @@ static void gfx_v10_0_check_fw_write_wait(struct amdgpu_device *adev)
 		    (adev->gfx.mec_feature_version >= 27))
 			adev->gfx.cp_fw_write_wait = true;
 		break;
+	case CHIP_SIENNA_CICHLID:
+		adev->gfx.cp_fw_write_wait = true;
+		break;
 	default:
 		break;
 	}
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
index ce2f72430403..061900e8afd3 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
@@ -350,6 +350,24 @@ static void gmc_v10_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid,
 	/* flush hdp cache */
 	adev->nbio.funcs->hdp_flush(adev, NULL);
 
+	/* For SRIOV run time, driver shouldn't access the register through MMIO
+	 * Directly use kiq to do the vm invalidation instead
+	 */
+	if (adev->gfx.kiq.ring.sched.ready &&
+	    (amdgpu_sriov_runtime(adev) || !amdgpu_sriov_vf(adev)) &&
+	    !adev->in_gpu_reset) {
+
+		struct amdgpu_vmhub *hub = &adev->vmhub[vmhub];
+		const unsigned eng = 17;
+		u32 inv_req = gmc_v10_0_get_invalidate_req(vmid, flush_type);
+		u32 req = hub->vm_inv_eng0_req + eng;
+		u32 ack = hub->vm_inv_eng0_ack + eng;
+
+		amdgpu_virt_kiq_reg_write_reg_wait(adev, req, ack, inv_req,
+				1 << vmid);
+		return;
+	}
+
 	mutex_lock(&adev->mman.gtt_window_lock);
 
 	if (vmhub == AMDGPU_MMHUB_0) {
-- 
2.25.4