[Intel-gfx] IOMMU RMRR for Intel graphic device

[Lu Baolu]

Forward to i915 mailing list and post the question again so people knows
what are the key concerns.

The background is that Linux community is going to collect and report
the reserved memory ranges of an assigned device to VFIO driver, and any
mapped address will be checked against the reserved ranges. If there's
any conflict, vifo will refuse the map request.

Unfortunately, when it comes Intel graphic device, the conflict happens.
That means address range mapped through vfio conflicts with the rmrr for
graphic device.

https://lkml.org/lkml/2018/6/5/760

The questions are 1) will the rmrr for graphic device still needs to be
reserved for BIOS or firmware use, when a device is going to be assigned
to user level? 2) if no, what's the check point, after which the rmrr is
unnecessary anymore?

Best regards,
Lu Baolu

On 5/9/19 5:14 PM, Zhang, Tina wrote:
> Hi Baolu,
> 
> +Xiong
> 
> I think the root cause is that guest i915 needs to access RMRR. Xiong cooked a patch to disable the RMRR use in i915 guest, however that patch didn't get landed into the i915 upstream repo due to some concerns from i915 maintainers. Xiong can give us more backgrounds.
> 
> So agreed, need to ask the i915 folk for this.
> 
> BR,
> Tina
> 
> 
>> -----Original Message-----
>> From: Yuan, Hang
>> Sent: Thursday, May 9, 2019 4:07 PM
>> To: Lu Baolu <baolu.lu at linux.intel.com>; Tian, Kevin <kevin.tian at intel.com>;
>> Zhenyu Wang <zhenyuw at linux.intel.com>; Zhang, Tina
>> <tina.zhang at intel.com>; Lu, Baolu <baolu.lu at intel.com>; Liu, Yi L
>> <yi.l.liu at intel.com>
>> Subject: RE: IOMMU RMRR for Intel graphic device
>>
>> Hi Baolu, as Kevin suggested, would you like to ask i915 people in their
>> mailing list intel-gfx at lists.freedesktop.org?
>>
>> Regards,
>> Henry
>>
>>> -----Original Message-----
>>> From: Lu Baolu [mailto:baolu.lu at linux.intel.com]
>>> Sent: Thursday, May 9, 2019 2:42 PM
>>> To: Tian, Kevin <kevin.tian at intel.com>; Zhenyu Wang
>>> <zhenyuw at linux.intel.com>; Zhang, Tina <tina.zhang at intel.com>; Yuan,
>>> Hang <hang.yuan at intel.com>; Lu, Baolu <baolu.lu at intel.com>; Liu, Yi L
>>> <yi.l.liu at intel.com>
>>> Cc: baolu.lu at linux.intel.com
>>> Subject: Re: IOMMU RMRR for Intel graphic device
>>>
>>> Hi,
>>>
>>> +Tina and Henry and cc more people
>>>
>>> The background is that Linux community is going to collect and report
>>> the reserved memory ranges of an assigned device to VFIO driver, and
>>> any mapped address will be checked against the reserved ranges. If
>>> there's any conflict, vifo will refuse the map request.
>>>
>>> Unfortunately, when it comes Intel graphic device, the conflict happens.
>>> That means address range mapped through vfio conflicts with the rmrr
>>> for graphic device.
>>>
>>> https://lkml.org/lkml/2018/6/5/760
>>>
>>> The questions are 1) will the rmrr for graphic device still needs to
>>> be reserved for BIOS or firmware use, when a device is going to be
>>> assigned to user level? 2) if no, what's the check point, after which
>>> the rmrr is unnecessary anymore?
>>>
>>> Best regards,
>>> Lu Baolu
>>>
>>> On 5/6/19 2:16 PM, Tian, Kevin wrote:
>>>> this should better be asked to i915 guys, since it's not
>>>> virtualization related. :-)
>>>>
>>>> One caveat, iirc, i915 driver tries to reuse stolen memory (covered
>>>> by
>>>> RMRR) even after boot time. take it as if another type of memory
>>>> resource. If true I'm afraid this might be a gap to your proposal.
>>>>
>>>> Since nothing confidential, possibly you can directly discuss in
>> community.
>>>>
>>>>> From: Lu Baolu [mailto:baolu.lu at linux.intel.com]
>>>>> Sent: Thursday, May 2, 2019 2:45 PM
>>>>>
>>>>> Ping...
>>>>>
>>>>> Any comments? This has been postponed in the community for a long
>>> time.
>>>>> We need to response this as soon as possible.
>>>>>
>>>>> Best regards,
>>>>> Lu Baolu
>>>>>
>>>>> On 4/29/19 1:19 PM, Lu Baolu wrote:
>>>>>> Hi Zhenyu,
>>>>>>
>>>>>> As we discussed, BIOS always exports IOMMU reserved memory
>> regions
>>>>>> for (a.k.a. RMRR in vt-d spec) Intel integrated graphic device.
>>>>>> This caused some problems when we pass-through such graphic
>>>>>> devices to
>>> user level.
>>>>>>
>>>>>> I am about to propose something to the community so that a RMRR
>>>>>> for graphic devices could be explicitly canceled as long as the
>>>>>> driver
>>>>>> (i915 or vfio) knows that the RMRR will never be used by BIOS
>> anymore.
>>>>>>
>>>>>> The same story happens for USB host controller devices. And since
>>>>>> we know that BIOS will stop using that memory region as soon as
>>>>>> the driver clears the SMI bits.
>>>>>>
>>>>>> So the question is, can graphic driver know when the RMRR for
>>>>>> graphic could be canceled?
>>>>>>
>>>>>> Best regards,
>>>>>> Lu Baolu
>>>>>>