[PATCH v4 4/6] drm/xe/pf: Force GuC virtualization mode

[Piotr Piórkowski]

Michal Wajdeczko <michal.wajdeczko at intel.com> wrote on pią [2025-lip-11 21:33:14 +0200]:
> By default the GuC starts in the 'native' mode and enables the VGT
> mode (aka 'virtualization' mode) only after it receives at least one
> set of VF configuration data. While this happens naturally while PF
> begins VFs provisioning, we might need this sooner as some actions,
> like TLB_INVALIDATION_ALL(0x7002), is supported by the GuC only in
> the VGT mode.
> 
> And this becomes a real problem if we would want to use above action
> to invalidate the LMTT early during VFs auto-provisioning, before VFs
> are enabled, as such H2G would be rejected:
> 
>  [ ] xe 0000:4d:00.0: [drm] *ERROR* GT0: FAST_REQ H2G fence 0x804e failed! e=0x30, h=0
>  [ ] xe 0000:4d:00.0: [drm] *ERROR* GT0: Fence 0x804e was used by action 0x7002 sent at:
>       h2g_write+0x33e/0x870 [xe]
>       __guc_ct_send_locked+0x1e1/0x1110 [xe]
>       guc_ct_send_locked+0x9f/0x740 [xe]
>       xe_guc_ct_send_locked+0x19/0x60 [xe]
>       send_tlb_invalidation+0xc2/0x470 [xe]
>       xe_gt_tlb_invalidation_all_async+0x45/0xa0 [xe]
>       xe_gt_tlb_invalidation_all+0x4b/0xa0 [xe]
>       lmtt_invalidate_hw+0x64/0x1a0 [xe]
>       xe_lmtt_invalidate_hw+0x5c/0x340 [xe]
>       pf_update_vf_lmtt+0x398/0xae0 [xe]
>       pf_provision_vf_lmem+0x350/0xa60 [xe]
>       xe_gt_sriov_pf_config_bulk_set_lmem+0xe2/0x410 [xe]
>       xe_gt_sriov_pf_config_set_fair_lmem+0x1c6/0x620 [xe]
>       xe_gt_sriov_pf_config_set_fair+0xd5/0x3f0 [xe]
>       xe_pci_sriov_configure+0x360/0x1200 [xe]
>       sriov_numvfs_store+0xbc/0x1d0
>       dev_attr_store+0x17/0x40
>       sysfs_kf_write+0x4a/0x80
>       kernfs_fop_write_iter+0x166/0x220
>       vfs_write+0x2ba/0x580
>       ksys_write+0x77/0x100
>       __x64_sys_write+0x19/0x30
>       x64_sys_call+0x2bf/0x2660
>       do_syscall_64+0x93/0x7a0
>       entry_SYSCALL_64_after_hwframe+0x76/0x7e
>  [ ] xe 0000:4d:00.0: [drm] *ERROR* GT0: CT dequeue failed: -71
>  [ ] xe 0000:4d:00.0: [drm] GT0: trying reset from receive_g2h [xe]
> 
> This could be mitigated by pushing earlier a PF self-configuration
> with some hard-coded values that cover unlimited access to the GGTT,
> use of all GuC contexts and doorbells.  This step is sufficient for
> the GuC to switch into the VGT mode.
> 
> Signed-off-by: Michal Wajdeczko <michal.wajdeczko at intel.com>
> ---
> v2: rebase and push min config as part of the full PF config
> ---
>  drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c | 26 ++++++++++++++++++++++
>  1 file changed, 26 insertions(+)
> 
> diff --git a/drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c b/drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c
> index 55afc5b06371..86dade6f1083 100644
> --- a/drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c
> +++ b/drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c
> @@ -341,6 +341,17 @@ static int pf_push_full_vf_config(struct xe_gt *gt, unsigned int vfid)
>  	}
>  	xe_gt_assert(gt, num_dwords <= max_cfg_dwords);
>  
> +	if (vfid == PFID) {
> +		u64 ggtt_start = xe_wopcm_size(gt_to_xe(gt));
> +		u64 ggtt_size = gt_to_tile(gt)->mem.ggtt->size - ggtt_start;
> +
> +		/* plain PF config data will never include a real GGTT region */
> +		xe_gt_assert(gt, !encode_config_ggtt(cfg + num_dwords, config, true));
> +
> +		/* fake PF GGTT config covers full GGTT range except reserved WOPCM */
> +		num_dwords += encode_ggtt(cfg + num_dwords, ggtt_start, ggtt_size, true);
> +	}
> +
>  	num_klvs = xe_guc_klv_count(cfg, num_dwords);
>  	err = pf_push_vf_buf_klvs(gt, vfid, num_klvs, buf, num_dwords);
>  
> @@ -2375,6 +2386,20 @@ int xe_gt_sriov_pf_config_restore(struct xe_gt *gt, unsigned int vfid,
>  	return err;
>  }
>  
> +static void pf_prepare_self_config(struct xe_gt *gt)
> +{
> +	struct xe_gt_sriov_config *config = pf_pick_vf_config(gt, PFID);
> +
> +	/*
> +	 * We want PF to be allowed to use all of context ID, doorbells IDs
> +	 * and whole usable GGTT area. While we can store ctxs/dbs numbers
> +	 * directly in the config structure, can't do the same with the GGTT
> +	 * configuration, so let it be prepared on demand while pushing KLVs.
> +	 */
> +	config->num_ctxs = GUC_ID_MAX;
> +	config->num_dbs = GUC_NUM_DOORBELLS;
> +}
> +
>  static int pf_push_self_config(struct xe_gt *gt)
>  {
>  	int err;
> @@ -2418,6 +2443,7 @@ int xe_gt_sriov_pf_config_init(struct xe_gt *gt)
>  	xe_gt_assert(gt, IS_SRIOV_PF(xe));
>  
>  	mutex_lock(xe_gt_sriov_pf_master_mutex(gt));
> +	pf_prepare_self_config(gt);
>  	err = pf_push_self_config(gt);
>  	mutex_unlock(xe_gt_sriov_pf_master_mutex(gt));

LGTM:
Reviewed-by: Piotr Piórkowski <piotr.piorkowski at intel.com>
>  
> -- 
> 2.47.1
> 

--