[Intel-gfx] [RFC PATCH] iommu/vt-d: Fix IOMMU field not populated on device hot re-plug

[Janusz Krzysztofik]

Hi Lu,

On Friday, August 23, 2019 3:51:11 AM CEST Lu Baolu wrote:
> Hi,
> 
> On 8/22/19 10:29 PM, Janusz Krzysztofik wrote:
> > When a perfectly working i915 device is hot unplugged (via sysfs) and
> > hot re-plugged again, its dev->archdata.iommu field is not populated
> > again with an IOMMU pointer.  As a result, the device probe fails on
> > DMA mapping error during scratch page setup.
> > 
> > It looks like that happens because devices are not detached from their
> > MMUIO bus before they are removed on device unplug.  Then, when an
> > already registered device/IOMMU association is identified by the
> > reinstantiated device's bus and function IDs on IOMMU bus re-attach
> > attempt, the device's archdata is not populated with IOMMU information
> > and the bad happens.
> > 
> > I'm not sure if this is a proper fix but it works for me so at least it
> > confirms correctness of my analysis results, I believe.  So far I
> > haven't been able to identify a good place where the possibly missing
> > IOMMU bus detach on device unplug operation could be added.
> 
> Which kernel version are you testing with? Does it contain below commit?
> 
> commit 458b7c8e0dde12d140e3472b80919cbb9ae793f4
> Author: Lu Baolu <baolu.lu at linux.intel.com>
> Date:   Thu Aug 1 11:14:58 2019 +0800

I was using an internal branch based on drm-tip which didn't contain this 
commit yet.  Fortunately it has been already merged into drm-tip over last 
weekend and has effectively fixed the issue.

Thanks,
Janusz

>      iommu/vt-d: Detach domain when move device out of group
> 
>      When removing a device from an iommu group, the domain should
>      be detached from the device. Otherwise, the stale domain info
>      will still be cached by the driver and the driver will refuse
>      to attach any domain to the device again.
> 
>      Cc: Ashok Raj <ashok.raj at intel.com>
>      Cc: Jacob Pan <jacob.jun.pan at linux.intel.com>
>      Cc: Kevin Tian <kevin.tian at intel.com>
>      Fixes: b7297783c2bb6 ("iommu/vt-d: Remove duplicated code for 
> device hotplug")
>      Reported-and-tested-by: Vlad Buslov <vladbu at mellanox.com>
>      Suggested-by: Robin Murphy <robin.murphy at arm.com>
>      Link: https://lkml.org/lkml/2019/7/26/1133
>      Signed-off-by: Lu Baolu <baolu.lu at linux.intel.com>
>      Signed-off-by: Joerg Roedel <jroedel at suse.de>
> 
> Best regards,
> Lu Baolu
> 
> > 
> > Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik at linux.intel.com>
> > ---
> >   drivers/iommu/intel-iommu.c | 3 +++
> >   1 file changed, 3 insertions(+)
> > 
> > diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
> > index 12d094d08c0a..7cdcd0595408 100644
> > --- a/drivers/iommu/intel-iommu.c
> > +++ b/drivers/iommu/intel-iommu.c
> > @@ -2477,6 +2477,9 @@ static struct dmar_domain 
*dmar_insert_one_dev_info(struct intel_iommu *iommu,
> >   		if (info2) {
> >   			found      = info2->domain;
> >   			info2->dev = dev;
> > +
> > +			if (dev && !dev->archdata.iommu)
> > +				dev->archdata.iommu = info2;
> >   		}
> >   	}
> >   
> > 
>