[Intel-gfx] [PATCH 1/4] iommu/vt-d: Disable superpage for Geminilake igfx

[Ville Syrjälä]

On Tue, Jul 13, 2021 at 09:34:09AM +0800, Lu Baolu wrote:
> On 7/12/21 11:47 PM, Ville Syrjälä wrote:
> > On Mon, Jul 12, 2021 at 07:23:07AM +0800, Lu Baolu wrote:
> >> On 7/10/21 12:47 AM, Ville Syrjala wrote:
> >>> From: Ville Syrjälä<ville.syrjala at linux.intel.com>
> >>>
> >>> While running "gem_exec_big --r single" from igt-gpu-tools on
> >>> Geminilake as soon as a 2M mapping is made I tend to get a DMAR
> >>> write fault. Strangely the faulting address is always a 4K page
> >>> and usually very far away from the 2M page that got mapped.
> >>> But if no 2M mappings get used I can't reproduce the fault.
> >>>
> >>> I also tried to dump the PTE for the faulting address but it actually
> >>> looks correct to me (ie. definitely seems to have the write bit set):
> >>>    DMAR: DRHD: handling fault status reg 2
> >>>    DMAR: [DMA Write] Request device [00:02.0] PASID ffffffff fault addr 7fa8a78000 [fault reason 05] PTE Write access is not set
> >>>    DMAR: fault 7fa8a78000 (level=1) PTE = 149efc003
> >>>
> >>> So not really sure what's going on and this might just be full on duct
> >>> tape, but it seems to work here. The machine has now survived a whole day
> >>> running that test whereas with superpage enabled it fails in less than
> >>> a minute usually.
> >>>
> >>> TODO: might be nice to disable superpage only for the igfx iommu
> >>>         instead of both iommus
> >> If all these quirks are about igfx dedicated iommu's, I would suggest to
> >> disable superpage only for the igfx ones.
> > Sure. Unfortunately there's no convenient mechanism to do that in
> > the iommu driver that I can immediately see. So not something I
> > can just whip up easily. Since you're actually familiar with the
> > driver maybe you can come up with a decent solution for that?
> > 
> 
> How about something like below? [no compile, no test...]
> 
> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> index 1131b8efb050..2d51ef288a9e 100644
> --- a/drivers/iommu/intel/iommu.c
> +++ b/drivers/iommu/intel/iommu.c
> @@ -338,6 +338,7 @@ static int intel_iommu_strict;
>   static int intel_iommu_superpage = 1;
>   static int iommu_identity_mapping;
>   static int iommu_skip_te_disable;
> +static int iommu_skip_igfx_superpage;
> 
>   #define IDENTMAP_GFX		2
>   #define IDENTMAP_AZALIA		4
> @@ -652,6 +653,27 @@ static bool domain_update_iommu_snooping(struct 
> intel_iommu *skip)
>   	return ret;
>   }
> 
> +static bool domain_use_super_page(struct dmar_domain *domain)
> +{
> +	struct dmar_drhd_unit *drhd;
> +	struct intel_iommu *iommu;
> +	bool ret = true;
> +
> +	if (!intel_iommu_superpage)
> +		return false;
> +
> +	rcu_read_lock();
> +	for_each_active_iommu(iommu, drhd) {
> +		if (drhd->gfx_dedicated && iommu_skip_igfx_superpage) {
> +			ret = false;
> +			break
                             ^
Missing semicolon. Othwerwise seems to work great here. Thanks.

Are you going to turn this into a proper patch, or do you
want me to just squash this into my patches and repost?

> +		}
> +	}
> +	rcu_read_unlock();
> +
> +	return ret;
> +}
> +
>   static int domain_update_iommu_superpage(struct dmar_domain *domain,
>   					 struct intel_iommu *skip)
>   {
> @@ -659,7 +681,7 @@ static int domain_update_iommu_superpage(struct 
> dmar_domain *domain,
>   	struct intel_iommu *iommu;
>   	int mask = 0x3;
> 
> -	if (!intel_iommu_superpage)
> +	if (!domain_use_super_page(domain))
>   		return 0;
> 
>   	/* set iommu_superpage to the smallest common denominator */
> @@ -5656,6 +5678,14 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 
> 0x1632, quirk_iommu_igfx);
>   DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163A, quirk_iommu_igfx);
>   DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163D, quirk_iommu_igfx);
> 
> +static void quirk_skip_igfx_superpage(struct pci_dev *dev)
> +{
> +	pci_info(dev, "Disabling IOMMU superpage for graphics on this chipset\n");
> +	iommu_skip_igfx_superpage = 1;
> +}
> +
> +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x3184, 
> quirk_skip_igfx_superpage);
> +
>   static void quirk_iommu_rwbf(struct pci_dev *dev)
>   {
>   	if (risky_device(dev))
> 
> Best regards,
> baolu

-- 
Ville Syrjälä
Intel