[Intel-gfx] [PATCH] drm/i915: Protect intel_engine_wakeup() for call from irq context

Mon Mar 20 14:39:39 UTC 2017

On 20/03/2017 14:31, Chris Wilson wrote:
> intel_engine_wakeup() is called by nop_request_submit() which is
> installed to handle third party fences completed from within irq
> context. As such, it needs the full irqsave/irqrestore and not the
> partial spin_irq_lock handling.
>
> [18942.714467] =================================
> [18942.719076] [ INFO: inconsistent lock state ]
> [18942.723522] 4.11.0-rc2-CI-CI_DRM_2368+ #1 Tainted: G     U  W
> [18942.729970] ---------------------------------
> [18942.734466] inconsistent {IN-SOFTIRQ-W} -> {SOFTIRQ-ON-W} usage.
> [18942.740594] gem_eio/1275 [HC0[0]:SC0[0]:HE1:SE1] takes:
> [18942.745932]  (&(&fence->lock)->rlock){+.?...}, at: [<ffffffff815ec100>] dma_fence_signal+0x100/0x
> 230
> [18942.755331] {IN-SOFTIRQ-W} state was registered at:
> [18942.760356]   __lock_acquire+0x5d0/0x1bb0
> [18942.764444]   lock_acquire+0xc9/0x220
> [18942.768196]   _raw_spin_lock_irqsave+0x41/0x60
> [18942.772747]   dma_fence_signal+0x100/0x230
> [18942.776927]   vgem_fence_timeout+0x9/0x10 [vgem]
> [18942.781701]   call_timer_fn+0x92/0x380
> [18942.785557]   expire_timers+0x150/0x1f0
> [18942.789491]   run_timer_softirq+0x7c/0x160
> [18942.793705]   __do_softirq+0x116/0x4c0
> [18942.797560]   irq_exit+0xa9/0xc0
> [18942.800873]   smp_apic_timer_interrupt+0x38/0x50
> [18942.805611]   apic_timer_interrupt+0x90/0xa0
> [18942.810008]   cpuidle_enter_state+0x135/0x380
> [18942.814503]   cpuidle_enter+0x12/0x20
> [18942.818250]   call_cpuidle+0x1e/0x40
> [18942.821906]   do_idle+0x17e/0x1f0
> [18942.825333]   cpu_startup_entry+0x18/0x20
> [18942.829463]   rest_init+0x127/0x130
> [18942.833025]   start_kernel+0x3f1/0x3fe
> [18942.836908]   x86_64_start_reservations+0x2a/0x2c
> [18942.841733]   x86_64_start_kernel+0x173/0x186
> [18942.846234]   verify_cpu+0x0/0xfc
> [18942.849604] irq event stamp: 30568
> [18942.853140] hardirqs last  enabled at (30567): [<ffffffff8110b81f>] ktime_get+0xef/0x120
> [18942.861468] hardirqs last disabled at (30568): [<ffffffff81876377>] _raw_spin_lock_irqsave+0x17/0
> x60
> [18942.870812] softirqs last  enabled at (30462): [<ffffffff81085cd9>] __do_softirq+0x1d9/0x4c0
> [18942.879443] softirqs last disabled at (30439): [<ffffffff81086139>] irq_exit+0xa9/0xc0
> [18942.887616]
> [18942.887616] other info that might help us debug this:
> [18942.894279]  Possible unsafe locking scenario:
> [18942.894279]
> [18942.900336]        CPU0
> [18942.902851]        ----
> [18942.905362]   lock(&(&fence->lock)->rlock);
> [18942.909647]   <Interrupt>
> [18942.912330]     lock(&(&fence->lock)->rlock);
> [18942.916821]
> [18942.916821]  *** DEADLOCK ***
> [18942.916821]
> [18942.922862] 1 lock held by gem_eio/1275:
> [18942.926859]  #0:  (&(&fence->lock)->rlock){+.?...}, at: [<ffffffff815ec100>] dma_fence_signal+0x1
> 00/0x230
> [18942.936651]
> [18942.936651] stack backtrace:
> [18942.941142] CPU: 3 PID: 1275 Comm: gem_eio Tainted: G     U  W       4.11.0-rc2-CI-CI_DRM_2368+ #
> 1
> [18942.950367] Hardware name: Gigabyte Technology Co., Ltd. Z170X-UD5/Z170X-UD5-CF, BIOS F21 01/06/2
> 017
> [18942.959756] Call Trace:
> [18942.962244]  dump_stack+0x67/0x92
> [18942.965626]  print_usage_bug.part.23+0x259/0x268
> [18942.970362]  mark_lock+0x12c/0x6f0
> [18942.973851]  ? check_usage_forwards+0x130/0x130
> [18942.978487]  mark_held_locks+0x6f/0xa0
> [18942.982329]  ? _raw_spin_unlock_irq+0x27/0x50
> [18942.986797]  trace_hardirqs_on_caller+0x150/0x200
> [18942.991599]  trace_hardirqs_on+0xd/0x10
> [18942.995515]  _raw_spin_unlock_irq+0x27/0x50
> [18942.999796]  intel_engine_wakeup+0x26/0x30 [i915]
> [18943.004670]  intel_engine_init_global_seqno+0x131/0x1a0 [i915]
> [18943.010745]  nop_submit_request+0x2e/0x40 [i915]
> [18943.015476]  submit_notify+0x3f/0x5c [i915]
> [18943.019763]  __i915_sw_fence_complete+0x176/0x220 [i915]
> [18943.025234]  ? try_to_del_timer_sync+0x4d/0x60
> [18943.029825]  i915_sw_fence_complete+0x25/0x40 [i915]
> [18943.034887]  dma_i915_sw_fence_wake+0x26/0x60 [i915]
> [18943.039959]  dma_fence_signal+0x146/0x230
> [18943.044109]  vgem_fence_signal_ioctl+0x6c/0xc0 [vgem]
> [18943.049275]  drm_ioctl+0x200/0x450
> [18943.052758]  ? vgem_fence_attach_ioctl+0x270/0x270 [vgem]
> [18943.058334]  do_vfs_ioctl+0x90/0x6e0
> [18943.061991]  ? entry_SYSCALL_64_fastpath+0x5/0xb1
> [18943.066843]  ? __this_cpu_preempt_check+0x13/0x20
> [18943.071643]  ? trace_hardirqs_on_caller+0xe7/0x200
> [18943.076532]  SyS_ioctl+0x3c/0x70
> [18943.079842]  entry_SYSCALL_64_fastpath+0x1c/0xb1
> [18943.084558] RIP: 0033:0x7f0dfcc14357
> [18943.088240] RSP: 002b:00007ffeb4628da8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
> [18943.095996] RAX: ffffffffffffffda RBX: ffffffff8147eb93 RCX: 00007f0dfcc14357
> [18943.103311] RDX: 00007ffeb4628de0 RSI: 0000000040086442 RDI: 0000000000000005
> [18943.110574] RBP: ffffc9000176ff88 R08: 0000000000000004 R09: 0000000000000000
> [18943.117845] R10: 0000000000000029 R11: 0000000000000246 R12: 0000000000000001
> [18943.125168] R13: 0000000000000005 R14: 0000000040086442 R15: 0000000000000000
> [18943.132520]  ? __this_cpu_preempt_check+0x13/0x20
>
> Signed-off-by: Chris Wilson <chris at chris-wilson.co.uk>
> Cc: Tvrtko Ursulin <tvrtko.ursulin at intel.com>
> Cc: Mika Kuoppala <mika.kuoppala at intel.com>
> ---
>  drivers/gpu/drm/i915/intel_breadcrumbs.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/intel_breadcrumbs.c b/drivers/gpu/drm/i915/intel_breadcrumbs.c
> index ba986edee312..b6ea192ad550 100644
> --- a/drivers/gpu/drm/i915/intel_breadcrumbs.c
> +++ b/drivers/gpu/drm/i915/intel_breadcrumbs.c
> @@ -47,11 +47,12 @@ static unsigned int __intel_breadcrumbs_wakeup(struct intel_breadcrumbs *b)
>  unsigned int intel_engine_wakeup(struct intel_engine_cs *engine)
>  {
>  	struct intel_breadcrumbs *b = &engine->breadcrumbs;
> +	unsigned long flags;
>  	unsigned int result;
>
> -	spin_lock_irq(&b->irq_lock);
> +	spin_lock_irqsave(&b->irq_lock, flags);
>  	result = __intel_breadcrumbs_wakeup(b);
> -	spin_unlock_irq(&b->irq_lock);
> +	spin_unlock_irqrestore(&b->irq_lock, flags);
>
>  	return result;
>  }
>

My bad, sorry!

Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin at intel.com>

Regards,

Tvrtko