[Intel-gfx] [core-for-CI][PATCH v2 0/2] iommu: Remove iova cpu hotplugging flushing

[Janusz Krzysztofik]

On Wednesday, 28 September 2022 11:22:21 CEST Tvrtko Ursulin wrote:
> 
> On 28/09/2022 09:58, Janusz Krzysztofik wrote:
> > Manual revert of commit f598a497bc7d ("iova: Add CPU hotplug handler to
> > flush rcaches").  It is trying to instantiate a cpuhp notifier from inside
> > a cpuhp callback.  That code replaced intel_iommu implementation of
> > flushing per-IOVA domain CPU rcaches which used a single instance of cpuhp
> > held for the module lifetime.
> > 
> > v2: Restore formerly reverted commit ac9a5d522bb8 "iommu/dma: Fix race
> >      condition during iova_domain initialization" after we have this more
> >      effective fix applied.  The former occurred not sufficient and is no
> >      longer needed once we have the latter in place.
> > 
> > Note: I'm no longer CCing IOMMU nor mainstream, they have been already
> >        warned.
> 
> Will that also fix the one below?

AFAICS, that one is supposed to be resolved by commit a349ffcb4daf 
("iommu/vt-d: Fix recursive lock issue in iommu_flush_dev_iotlb()").

Thanks,
Janusz

> 
> <4> [18.297089] ================================
> <4> [18.297090] WARNING: inconsistent lock state
> <4> [18.297092] 6.0.0-rc2-CI_DINF_262-g320305923c88+ #1 Not tainted
> <4> [18.297095] --------------------------------
> <4> [18.297096] inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
> <4> [18.297098] swapper/0/0 [HC0[0]:SC1[1]:HE1:SE0] takes:
> <4> [18.297101] ffff8881030ed678 (&domain->lock){+.?.}-{2:2}, at: 
iommu_flush_dev_iotlb.part.61+0x23/0x80
> <4> [18.297108] {SOFTIRQ-ON-W} state was registered at:
> <4> [18.297110]   lock_acquire+0xd3/0x310
> <4> [18.297113]   _raw_spin_lock+0x2a/0x40
> <4> [18.297116]   domain_update_iommu_cap+0x20b/0x2c0
> <4> [18.297118]   intel_iommu_attach_device+0x5bd/0x860
> <4> [18.297121]   __iommu_attach_device+0x18/0xe0
> <4> [18.297124]   bus_iommu_probe+0x1f3/0x2d0
> <4> [18.297126]   bus_set_iommu+0x82/0xd0
> <4> [18.297129]   intel_iommu_init+0xe45/0x102a
> <4> [18.297132]   pci_iommu_init+0x9/0x31
> <4> [18.297135]   do_one_initcall+0x53/0x2f0
> <4> [18.297138]   kernel_init_freeable+0x18f/0x1e1
> <4> [18.297140]   kernel_init+0x11/0x120
> <4> [18.297143]   ret_from_fork+0x1f/0x30
> <4> [18.297145] irq event stamp: 512712
> <4> [18.297147] hardirqs last  enabled at (512712): [<ffffffff81b59254>] 
_raw_spin_unlock_irqrestore+0x54/0x70
> <4> [18.297151] hardirqs last disabled at (512711): [<ffffffff81b58ffb>] 
_raw_spin_lock_irqsave+0x4b/0x50
> <4> [18.297155] softirqs last  enabled at (512700): [<ffffffff81e00323>] 
__do_softirq+0x323/0x48e
> <4> [18.297158] softirqs last disabled at (512705): [<ffffffff810c16a8>] 
irq_exit_rcu+0xb8/0xe0
> <4> [18.297162]
> other info that might help us debug this:
> <4> [18.297164]  Possible unsafe locking scenario:
> <4> [18.297166]        CPU0
> <4> [18.297168]        ----
> <4> [18.297169]   lock(&domain->lock);
> <4> [18.297171]   <Interrupt>
> <4> [18.297172]     lock(&domain->lock);
> <4> [18.297174]
>   *** DEADLOCK ***
> <4> [18.297176] 1 lock held by swapper/0/0:
> <4> [18.297177]  #0: ffffc90000003ea8 ((&cookie->fq_timer)){+.-.}-{0:0}, at: 
call_timer_fn+0x0/0x2c0
> <4> [18.297183]
> stack backtrace:
> <4> [18.297185] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.0.0-rc2-
CI_DINF_262-g320305923c88+ #1
> <4> [18.297189] Hardware name: ASUS System Product Name/ROG STRIX B660-I 
GAMING WIFI, BIOS 0415 09/29/2021
> <4> [18.297192] Call Trace:
> <4> [18.297193]  <IRQ>
> <4> [18.297195]  dump_stack_lvl+0x56/0x7f
> <4> [18.297198]  mark_lock.part.47+0x8d1/0xdb0
> <4> [18.297200]  ? qi_submit_sync+0x3c8/0x650
> <4> [18.297203]  __lock_acquire+0x861/0xb90
> <4> [18.297205]  lock_acquire+0xd3/0x310
> <4> [18.297207]  ? iommu_flush_dev_iotlb.part.61+0x23/0x80
> <4> [18.297211]  ? iommu_dma_map_sg+0x400/0x400
> <4> [18.297213]  _raw_spin_lock+0x2a/0x40
> <4> [18.297215]  ? iommu_flush_dev_iotlb.part.61+0x23/0x80
> <4> [18.297218]  iommu_flush_dev_iotlb.part.61+0x23/0x80
> <4> [18.297220]  intel_flush_iotlb_all+0x95/0x110
> <4> [18.297223]  fq_flush_iotlb+0x1d/0x30
> <4> [18.297225]  fq_flush_timeout+0x28/0xc0
> <4> [18.297227]  ? iommu_dma_map_sg+0x400/0x400
> <4> [18.297229]  ? iommu_dma_map_sg+0x400/0x400
> <4> [18.297231]  call_timer_fn+0x9c/0x2c0
> <4> [18.297234]  run_timer_softirq+0x548/0x570
> <4> [18.297237]  __do_softirq+0xda/0x48e
> <4> [18.297240]  irq_exit_rcu+0xb8/0xe0
> <4> [18.297242]  sysvec_apic_timer_interrupt+0x9e/0xc0
> <4> [18.297245]  </IRQ>
> 
> Regards,
> 
> Tvrtko
> 
> > 
> > 
> > Chris Wilson (1):
> >    iommu: Remove iova cpu hotplugging flushing
> > 
> > Janusz Krzysztofik (1):
> >    Restore "iommu/dma: Fix race condition during iova_domain
> >      initialization"
> > 
> >   drivers/iommu/dma-iommu.c  | 17 +++++++++++++----
> >   drivers/iommu/iova.c       | 28 ----------------------------
> >   include/linux/cpuhotplug.h |  1 -
> >   include/linux/iova.h       |  1 -
> >   4 files changed, 13 insertions(+), 34 deletions(-)
> > 
>