[Intel-gfx] [PATCH v2] iommu/intel: Exclude devices using RMRRs from IOMMU API domains

[Alex Williamson]

On Tue, 2014-06-17 at 13:41 +0100, David Woodhouse wrote:
> On Tue, 2014-06-17 at 06:22 -0600, Alex Williamson wrote:
> > On Tue, 2014-06-17 at 08:04 +0100, David Woodhouse wrote:
> > > On Mon, 2014-06-16 at 23:35 -0600, Alex Williamson wrote:
> > > > 
> > > > Any idea what an off-the-shelf Asus motherboard would be doing with an
> > > > RMRR on the Intel HD graphics?
> > > > 
> > > > dmar: RMRR base: 0x000000bb800000 end: 0x000000bf9fffff
> > > > IOMMU: Setting identity map for device 0000:00:02.0 [0xbb800000 - 0xbf9fffff]
> > > 
> > > Hm, we should have thought of that sooner. That's quite normal — it's
> > > for the 'stolen' memory used for the framebuffer. And maybe also the
> > > GTT, and shadow GTT and other things; I forget precisely what, and it
> > > varies from one setup to another.
> > 
> > Why exactly do these things need to be identity mapped through the
> > IOMMU?  This sounds like something a normal device might do with a
> > coherent mapping.
> 
> The BIOS (EFI or VESA) sets up a framebuffer in stolen main memory. It's
> accessed by DMA, using the physical address. The RMRR exists because we
> need it *not* to suddenly stop working the moment the OS turns on the
> IOMMU.
> 
> The OS graphics driver, if any, is not loaded at this point.
> 
> And even later, the OS graphics driver may choose to make use of the
> 'stolen' memory for various purposes. And since it was already stolen,
> it doesn't go and set up *another* mapping for it; it knows that a
> mapping already exists.
> 
> > > I'd expect fairly much all systems to have an RMRR for the integrated
> > > graphics device if they have one, and your patch¹ is going to prevent
> > > assignment of those to guests... as you've presumably noticed.
> > > 
> > > I'm not sure if the i915 driver is capable of fully reprogramming the
> > > hardware to completely stop using that region, to allow assignment to a
> > > guest with a 'pure' memory map and no stolen region. I suppose it must,
> > > if assignment to guests was working correctly before?
> > 
> > IGD assignment has never worked with KVM.
> 
> Hm. It works with Xen though, doesn't it?

Apparently

> Are we content to say that it'll *never* work with KVM, and thus we can
> live with the fact that your patch makes it harder to fix whatever was
> wrong in the first place?

Probably not.  However, it seems like you're saying that this RMRR is
used by and visible to OS level drivers, versus backchannel
communication channels, invisible to the OS.  I think the latter is
specifically what we want to prevent by excluding devices with RMRRs.
This is a challenging use case, but it seems to be understood.  If when
IGD is bound to vfio-pci we can be sure that access to the RMRR area
ceases, then we can tear it down and re-establish it from
userspace/QEMU, describe it to the guest in an e820 reserved region, and
never consider hotplug of the device for guests.  If that's the case,
maybe it's another exception, like USB.  I'll need to look through i915
more to find how the region is discovered.  Thanks,

Alex