On Tue, Feb 26, 2019 at 11:02:48AM +0100, Lucas Stach wrote:
> Hi Russell,
>
> Am Dienstag, den 26.02.2019, 09:24 +0000 schrieb Russell King - ARM Linux admin:
> > I'm not sure when this happened, only that it happened sometime
> > overnight. It was left running an Xfce desktop having only logged in,
> > but with the monitor disconnected. Fedora 23 plus xf86-video-armada.
> >
> > I don't have any more information than that and the kernel messages
> > below.
>
> Thanks for the report! Both of those issues are caused by the GPU
> scheduler failing to put the scheduler fence callback execution into a
> work item, like it normally does, in the dying sched_entity cleanup.
> This causes multiple code paths which expect to be called from a clean
> process context to be called from the same IRQ context with a number of
> locks potentially already held.
>
> It will take me some time to work through the corners of this code, but
> I should have a patch fixing this later today.
Hi Lucas,
Did you get a chance to patch this bug?
Thanks.
>
> Regards,
> Lucas
>
> > [51328.909729] ============================================
> > [51328.915044] WARNING: possible recursive locking detected
> > [51328.920361] 4.20.0+ #307 Not tainted
> > [51328.923939] --------------------------------------------
> > [51328.929254] Xorg/5379 is trying to acquire lock:
> > [51328.933874] cd12d5e4 (&(&fence->lock)->rlock){-.-.}, at: dma_fence_signal+0x9c/0x1d4
> > [51328.941638]
> > [51328.941638] but task is already holding lock:
> > [51328.947474] cd12d6a4 (&(&fence->lock)->rlock){-.-.}, at: dma_fence_signal+0x9c/0x1d4
> > [51328.955226]
> > [51328.955226] other info that might help us debug this:
> > [51328.961756] Possible unsafe locking scenario:
> > [51328.961756]
> > [51328.967678] CPU0
> > [51328.970127] ----
> > [51328.976761] lock(&(&fence->lock)->rlock);
> > [51328.980948]
> > [51328.980948] *** DEADLOCK ***
> > [51328.980948]
> > [51328.986871] May be due to missing lock nesting notation
> > [51328.986871]
> > [51328.993663] 4 locks held by Xorg/5379:
> > [51328.997414] #0: d8c6bdd0 (reservation_ww_class_acquire){+.+.}, at: drm_ioctl_kernel+0x90/0xc8
> > [51329.006040] #1: d879e2ac (&suballoc->lock){+.+.}, at: etnaviv_cmdbuf_init+0x5c/0x16c [etnaviv]
> > [51329.014784] #2: d900dd88 (&(&gpu->fence_spinlock)->rlock){-.-.}, at: dma_fence_signal+0x9c/0x1d4
> > [51329.023668] #3: cd12d6a4 (&(&fence->lock)->rlock){-.-.}, at: dma_fence_signal+0x9c/0x1d4
> > [51329.031854]
> > [51329.031854] stack backtrace:
> > [51329.036218] CPU: 0 PID: 5379 Comm: Xorg Not tainted 4.20.0+ #307
> > [51329.042227] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
> > [51329.048772] [<c0019658>] (unwind_backtrace) from [<c001489c>] (show_stack+0x10/0x14)
> > [51329.056527] [<c001489c>] (show_stack) from [<c07fb420>] (dump_stack+0x9c/0xd4)
> > [51329.063763] [<c07fb420>] (dump_stack) from [<c0086818>] (__lock_acquire+0x1270/0x19b8)
> > [51329.071689] [<c0086818>] (__lock_acquire) from [<c0087848>] (lock_acquire+0xc4/0x1dc)
> > [51329.079529] [<c0087848>] (lock_acquire) from [<c081a0a4>] (_raw_spin_lock_irqsave+0x44/0x58)
> > [51329.087975] [<c081a0a4>] (_raw_spin_lock_irqsave) from [<c04c4dec>] (dma_fence_signal+0x9c/0x1d4)
> > [51329.096872] [<c04c4dec>] (dma_fence_signal) from [<bf046908>] (drm_sched_entity_kill_jobs_cb+0x14/0x78 [gpu_sched])
> > [51329.107327] [<bf046908>] (drm_sched_entity_kill_jobs_cb [gpu_sched]) from [<c04c4e30>] (dma_fence_signal+0xe0/0x1d4)
> > [51329.117866] [<c04c4e30>] (dma_fence_signal) from [<bf0458ec>] (drm_sched_process_job+0x60/0x1a4 [gpu_sched])
> > [51329.127710] [<bf0458ec>] (drm_sched_process_job [gpu_sched]) from [<c04c4e30>] (dma_fence_signal+0xe0/0x1d4)
> > [51329.137569] [<c04c4e30>] (dma_fence_signal) from [<bf05ed58>] (irq_handler+0xc8/0x1d8 [etnaviv])
> > [51329.146389] [<bf05ed58>] (irq_handler [etnaviv]) from [<c009bd98>] (__handle_irq_event_percpu+0x98/0x378)
> > [51329.155966] [<c009bd98>] (__handle_irq_event_percpu) from [<c009c094>] (handle_irq_event_percpu+0x1c/0x58)
> > [51329.165629] [<c009c094>] (handle_irq_event_percpu) from [<c009c108>] (handle_irq_event+0x38/0x5c)
> > [51329.174511] [<c009c108>] (handle_irq_event) from [<c009f830>] (handle_fasteoi_irq+0x94/0x124)
> > [51329.183044] [<c009f830>] (handle_fasteoi_irq) from [<c009b4f8>] (generic_handle_irq+0x18/0x28)
> > [51329.191665] [<c009b4f8>] (generic_handle_irq) from [<c009b5fc>] (__handle_domain_irq+0x54/0xb0)
> > [51329.200374] [<c009b5fc>] (__handle_domain_irq) from [<c03a6afc>] (gic_handle_irq+0x48/0x98)
> > [51329.208735] [<c03a6afc>] (gic_handle_irq) from [<c0009a10>] (__irq_svc+0x70/0x98)
> > [51329.216221] Exception stack(0xd8c6bc28 to 0xd8c6bc70)
> > [51329.221279] bc20: 00000001 00000011 d8c6bc78 c2005940 00000000 d879e2ac
> > [51329.229462] bc40: 00000000 00000000 00000000 c0c2c580 600b0013 c0b98568 c13bb508 d8c6bc78
> > [51329.237643] bc60: c0085804 c0087860 800b0013 ffffffff
> > [51329.242704] [<c0009a10>] (__irq_svc) from [<c0087860>] (lock_acquire+0xdc/0x1dc)
> > [51329.250111] [<c0087860>] (lock_acquire) from [<c0815cc8>] (__mutex_lock+0x50/0x8b8)
> > [51329.257777] [<c0815cc8>] (__mutex_lock) from [<c08165b8>] (mutex_lock_nested+0x1c/0x24)
> > [51329.265812] [<c08165b8>] (mutex_lock_nested) from [<bf05a8e0>] (etnaviv_cmdbuf_init+0x5c/0x16c [etnaviv])
> > [51329.275424] [<bf05a8e0>] (etnaviv_cmdbuf_init [etnaviv]) from [<bf05c514>] (etnaviv_ioctl_gem_submit+0x61c/0x1210 [etnaviv])
> > [51329.286668] [<bf05c514>] (etnaviv_ioctl_gem_submit [etnaviv]) from [<c0460efc>] (drm_ioctl_kernel+0x90/0xc8)
> > [51329.296503] [<c0460efc>] (drm_ioctl_kernel) from [<c0461114>] (drm_ioctl+0x1e0/0x3b0)
> > [51329.304344] [<c0461114>] (drm_ioctl) from [<c01cae20>] (do_vfs_ioctl+0x90/0xa48)
> > [51329.311750] [<c01cae20>] (do_vfs_ioctl) from [<c01cb80c>] (ksys_ioctl+0x34/0x60)
> > [51329.319155] [<c01cb80c>] (ksys_ioctl) from [<c0009000>] (ret_fast_syscall+0x0/0x28)
> > [51329.326813] Exception stack(0xd8c6bfa8 to 0xd8c6bff0)
> > [51329.331870] bfa0: 00b0e120 be9f7b28 00000009 c0306446 be9f7b28 c0306400
> > [51329.340053] bfc0: 00b0e120 be9f7b28 c0306446 00000036 be9f7b84 be9f7b28 00e19a30 00e460c0
> > [51329.348234] bfe0: b6d19f18 be9f7ae4 b6d0185c b6a0be5c
> > [51329.353376] ------------[ cut here ]------------
> > [51329.358004] kernel BUG at /home/rmk/git/linux-rmk/mm/vmalloc.c:1612!
> > [51329.364365] Internal error: Oops - BUG: 0 [#1] SMP ARM
> > [51329.369510] Modules linked in: ip6t_REJECT nf_reject_ipv6 ip6t_rpfilter xt_tcpudp ipt_REJECT nf_reject_ipv4 xt_conntrack ip_set nfnetlink ebtable_broute ebtable_nat ip6table_raw ip6table_nat nf_nat_ipv6 ip6table_mangle iptable_raw iptable_nat nf_nat_ipv4 nf_nat nf_conntrack nf_defrag_ipv4 nf_defrag_ipv6 libcrc32c iptable_mangle ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter caam_jr error snd_soc_imx_spdif imx_thermal nvmem_imx_ocotp snd_soc_imx_audmux snd_soc_sgtl5000 caam imx_sdma virt_dma rc_cec coda snd_soc_fsl_spdif snd_soc_fsl_ssi v4l2_mem2mem imx_pcm_dma imx_vdoa videobuf2_dma_contig dw_hdmi_cec dw_hdmi_ahb_audio etnaviv gpu_sched imx6q_cpufreq sch_fq_codel nfsd ip_tables x_tables
> > [51329.432047] CPU: 0 PID: 5379 Comm: Xorg Not tainted 4.20.0+ #307
> > [51329.438059] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
> > [51329.444599] PC is at vunmap+0x48/0x4c
> > [51329.448298] LR is at etnaviv_gem_shmem_release+0x10/0x64 [etnaviv]
> > [51329.454486] pc : [<c0192220>] lr : [<bf05d4c8>] psr: 200b0193
> > [51329.460758] sp : d8c6ba90 ip : c23c4e98 fp : c0c2c314
> > [51329.465988] r10: 00000200 r9 : 00000100 r8 : c23c4f20
> > [51329.471219] r7 : c23c4e00 r6 : c9dc4600 r5 : 00000000 r4 : 00010000
> > [51329.477751] r3 : 00010000 r2 : d8c6ba90 r1 : c2005940 r0 : 00000000
> > [51329.484287] Flags: nzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment none
> > [51329.491514] Control: 10c5387d Table: 13340059 DAC: 00000051
> > [51329.497267] Process Xorg (pid: 5379, stack limit = 0xd8c6a218)
> > [51329.503105] Stack: (0xd8c6ba90 to 0xd8c6c000)
> > [51329.507470] ba80: bf05d4b8 c23c4e00 c23c4f8c bf05d4c8
> > [51329.515657] baa0: c23c4f8c bf05e258 d900e800 c23c4e00 c9c13a00 c23c4e00 c9c13a90 00000000
> > [51329.523843] bac0: c9c13a58 c045fab8 00000001 00000006 c9c13a38 bf05bdbc c9c13a0c cd12d940
> > [51329.532028] bae0: c9c13a00 cd12d650 200b0193 00000000 00000001 bf046944 d8df790c cd12d640
> > [51329.540214] bb00: d8df790c c04c4e30 d900d908 dba27308 cd12d640 cd12d600 bf047e84 bf0458ec
> > [51329.548400] bb20: dba27318 dba27308 dba27318 dba27318 200b0193 00000000 00000001 c04c4e30
> > [51329.556586] bb40: 00000002 d900d840 00000000 d900db20 d900dd98 bf05ed58 c9f94680 da962600
> > [51329.564772] bb60: c0b9efd4 00000016 00000000 d8c6bbb4 c0b98568 c009bd98 da962674 c0083798
> > [51329.572958] bb80: da962664 da962600 c2005940 da962600 da962600 c0b9efd4 00000029 db410400
> > [51329.581143] bba0: f4001100 d8c6bc28 c0b98568 c009c094 db410400 00000000 da962664 da962600
> > [51329.589329] bbc0: da962664 c009c108 da962600 da962664 c0b9efd4 c009f830 c0b8d9e4 00000000
> > [51329.597515] bbe0: 00000000 c009b4f8 0000012f c009b5fc f4000100 f400010c c0b98cf8 000003eb
> > [51329.605701] bc00: c0be9f4c c03a6afc c0087860 800b0013 ffffffff d8c6bc5c 00000000 d8c6a000
> > [51329.613887] bc20: 600b0013 c0009a10 00000001 00000011 d8c6bc78 c2005940 00000000 d879e2ac
> > [51329.622073] bc40: 00000000 00000000 00000000 c0c2c580 600b0013 c0b98568 c13bb508 d8c6bc78
> > [51329.630258] bc60: c0085804 c0087860 800b0013 ffffffff 00000051 bf000000 00000001 00000000
> > [51329.638444] bc80: 00000000 bf05a8e0 00000000 00000000 00000000 00000000 d879e2d8 d879e278
> > [51329.646629] bca0: 00000000 00000000 c13bb508 00000003 00000002 d879e2bc d9116f74 c0815cc8
> > [51329.654815] bcc0: 00000001 00000000 bf05a8e0 00000004 00000006 c2005940 d879e2ac 00000000
> > [51329.663001] bce0: d879e2bc c0087b6c 00000001 c2005940 c081a5f4 00000001 d879e278 000009c4
> > [51329.671187] bd00: d879e200 d879e2c8 d879e278 00000003 00000000 d879e2bc d9116f74 c08165b8
> > [51329.679373] bd20: bf05a8e0 d879e200 d879e2c8 bf05a8e0 00000000 00000000 c2005940 c007b8bc
> > [51329.687558] bd40: d8c6bd40 d8c6bd40 c0460efc c2ccb340 c716e000 00000010 d2fa8000 c9e5cc00
> > [51329.695743] bd60: 00000000 d900e040 d8c6be68 bf05c514 00000001 00000000 c0460efc 00000000
> > [51329.703929] bd80: c0bfe0f8 00000001 ffffffff 00000000 c9e5c000 00000000 d9116f00 c0087848
> > [51329.712115] bda0: 00000000 00000000 00000000 c0462190 c0bfaa0c c2005940 003cb56c 00000000
> > [51329.720300] bdc0: 00010000 00000000 c0bfe0f8 00000000 c0bfe0fc c0d97fc0 00000000 c0a3e038
> > [51329.728486] bde0: 00000001 0000000c c9e5cc00 d900e800 00000000 c9e5cc00 00000021 bf05bef8
> > [51329.736671] be00: d8c6be68 d8c6be68 d8c6be68 c0460efc be9f7b28 00000000 be9f7b28 00000030
> > [51329.744857] be20: c9e5cc00 be9f7b28 bf0636c0 c0306446 c307c640 c0461114 00000001 bf0649fc
> > [51329.753043] be40: c0c2c580 60070093 c13cc100 00000051 00000046 00000030 00000048 bf05bef8
> > [51329.761228] be60: c2005e48 00000000 00000000 00000001 00000001 00000003 0000008e 00006a90
> > [51329.769413] be80: 00ec5cc0 00000000 00ec6dc0 00000000 00b22048 00000000 00000000 00000000
> > [51329.777598] bea0: 00000000 00000000 00000000 00000000 c2008978 00000000 c13cc144 00000000
> > [51329.785784] bec0: 00000000 00000002 c0c2c580 60070093 c13cc100 00000000 c13cc144 c0083798
> > [51329.793969] bee0: c13cc100 c13cc144 c2005940 c01cb80c c307c640 00000009 00000009 be9f7b28
> > [51329.802155] bf00: c9eef568 be9f7b28 00e460c0 c01cae20 c13cc100 c00c2108 00000000 c00ba828
> > [51329.810340] bf20: 00000001 00000000 c00c2108 c00871a4 da2cf27c 00000023 0000c881 00000000
> > [51329.818526] bf40: 00000000 14e4373b a38c2bbe 00000051 00206a48 00000107 c00091c4 be9f7c0c
> > [51329.826712] bf60: 00000008 00000000 be9f7c0c c307c640 c307c640 c0306446 00000009 be9f7b28
> > [51329.834898] bf80: d8c6a000 00000036 00e460c0 c01cb80c 00b0e120 be9f7b28 c0306446 00000036
> > [51329.843084] bfa0: c00091c4 c0009000 00b0e120 be9f7b28 00000009 c0306446 be9f7b28 c0306400
> > [51329.851270] bfc0: 00b0e120 be9f7b28 c0306446 00000036 be9f7b84 be9f7b28 00e19a30 00e460c0
> > [51329.859456] bfe0: b6d19f18 be9f7ae4 b6d0185c b6a0be5c 80070010 00000009 00000000 00000000
> > [51329.867671] [<c0192220>] (vunmap) from [<bf05d4c8>] (etnaviv_gem_shmem_release+0x10/0x64 [etnaviv])
> > [51329.876770] [<bf05d4c8>] (etnaviv_gem_shmem_release [etnaviv]) from [<bf05e258>] (etnaviv_gem_free_object+0xd0/0x1c4 [etnaviv])
> > [51329.888279] [<bf05e258>] (etnaviv_gem_free_object [etnaviv]) from [<c045fab8>] (drm_gem_object_free+0x20/0x64)
> > [51329.898311] [<c045fab8>] (drm_gem_object_free) from [<bf05bdbc>] (etnaviv_submit_put+0x6c/0x1a8 [etnaviv])
> > [51329.908007] [<bf05bdbc>] (etnaviv_submit_put [etnaviv]) from [<bf046944>] (drm_sched_entity_kill_jobs_cb+0x50/0x78 [gpu_sched])
> > [51329.919506] [<bf046944>] (drm_sched_entity_kill_jobs_cb [gpu_sched]) from [<c04c4e30>] (dma_fence_signal+0xe0/0x1d4)
> > [51329.930048] [<c04c4e30>] (dma_fence_signal) from [<bf0458ec>] (drm_sched_process_job+0x60/0x1a4 [gpu_sched])
> > [51329.939896] [<bf0458ec>] (drm_sched_process_job [gpu_sched]) from [<c04c4e30>] (dma_fence_signal+0xe0/0x1d4)
> > [51329.949755] [<c04c4e30>] (dma_fence_signal) from [<bf05ed58>] (irq_handler+0xc8/0x1d8 [etnaviv])
> > [51329.958577] [<bf05ed58>] (irq_handler [etnaviv]) from [<c009bd98>] (__handle_irq_event_percpu+0x98/0x378)
> > [51329.968157] [<c009bd98>] (__handle_irq_event_percpu) from [<c009c094>] (handle_irq_event_percpu+0x1c/0x58)
> > [51329.977822] [<c009c094>] (handle_irq_event_percpu) from [<c009c108>] (handle_irq_event+0x38/0x5c)
> > [51329.986706] [<c009c108>] (handle_irq_event) from [<c009f830>] (handle_fasteoi_irq+0x94/0x124)
> > [51329.995242] [<c009f830>] (handle_fasteoi_irq) from [<c009b4f8>] (generic_handle_irq+0x18/0x28)
> > [51330.003866] [<c009b4f8>] (generic_handle_irq) from [<c009b5fc>] (__handle_domain_irq+0x54/0xb0)
> > [51330.012577] [<c009b5fc>] (__handle_domain_irq) from [<c03a6afc>] (gic_handle_irq+0x48/0x98)
> > [51330.020939] [<c03a6afc>] (gic_handle_irq) from [<c0009a10>] (__irq_svc+0x70/0x98)
> > [51330.028427] Exception stack(0xd8c6bc28 to 0xd8c6bc70)
> > [51330.033487] bc20: 00000001 00000011 d8c6bc78 c2005940 00000000 d879e2ac
> > [51330.041673] bc40: 00000000 00000000 00000000 c0c2c580 600b0013 c0b98568 c13bb508 d8c6bc78
> > [51330.049857] bc60: c0085804 c0087860 800b0013 ffffffff
> > [51330.054922] [<c0009a10>] (__irq_svc) from [<c0087860>] (lock_acquire+0xdc/0x1dc)
> > [51330.062332] [<c0087860>] (lock_acquire) from [<c0815cc8>] (__mutex_lock+0x50/0x8b8)
> > [51330.070000] [<c0815cc8>] (__mutex_lock) from [<c08165b8>] (mutex_lock_nested+0x1c/0x24)
> > [51330.078039] [<c08165b8>] (mutex_lock_nested) from [<bf05a8e0>] (etnaviv_cmdbuf_init+0x5c/0x16c [etnaviv])
> > [51330.087654] [<bf05a8e0>] (etnaviv_cmdbuf_init [etnaviv]) from [<bf05c514>] (etnaviv_ioctl_gem_submit+0x61c/0x1210 [etnaviv])
> > [51330.098900] [<bf05c514>] (etnaviv_ioctl_gem_submit [etnaviv]) from [<c0460efc>] (drm_ioctl_kernel+0x90/0xc8)
> > [51330.108738] [<c0460efc>] (drm_ioctl_kernel) from [<c0461114>] (drm_ioctl+0x1e0/0x3b0)
> > [51330.116579] [<c0461114>] (drm_ioctl) from [<c01cae20>] (do_vfs_ioctl+0x90/0xa48)
> > [51330.123986] [<c01cae20>] (do_vfs_ioctl) from [<c01cb80c>] (ksys_ioctl+0x34/0x60)
> > [51330.131393] [<c01cb80c>] (ksys_ioctl) from [<c0009000>] (ret_fast_syscall+0x0/0x28)
> > [51330.139055] Exception stack(0xd8c6bfa8 to 0xd8c6bff0)
> > [51330.144116] bfa0: 00b0e120 be9f7b28 00000009 c0306446 be9f7b28 c0306400
> > [51330.152302] bfc0: 00b0e120 be9f7b28 c0306446 00000036 be9f7b84 be9f7b28 00e19a30 00e460c0
> > [51330.160486] bfe0: b6d19f18 be9f7ae4 b6d0185c b6a0be5c
> > [51330.165548] Code: e1a00005 e1a01004 e8bd4038 eaffffb1 (e7f001f2)
> > [51330.171657] ---[ end trace d9abb4e69be36a96 ]---
> > [51330.176282] Kernel panic - not syncing: Fatal exception in interrupt
> > [51330.182654] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
> >
>
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up