[PATCH 1/2] drm/panfrost: mmu: improved memory attributes
Alexey Sheplyakov
asheplyakov at basealt.ru
Wed Jan 12 18:18:29 UTC 2022
Hi, Robin,
On Fri, Dec 24, 2021 at 12:56:57PM +0000, Robin Murphy wrote:
> I'd note that panfrost has been working OK - to the extent that Mesa
> supports its older ISA - on the T624 (single core group) in Arm's Juno SoC
> for over a year now since commit 268af50f38b1.
>
> If you have to force outer non-cacheable to avoid getting translation faults
> and other errors that look like the GPU is inexplicably seeing the wrong
> data, I'd check whether you have the same thing where your integration is
> actually I/O-coherent and you're missing the "dma-coherent" property in your
> DT.
Thanks for a detailed explanation. Indeed adding the "dma-coherent" property
(and the 2nd patch which effectively disables the 2nd core group) is enough
to get panfrost working with T628 on BE-M1000 SoC. Unfortunately the same
trick does NOT work for Exynos 5422. Here I get an Oops as soon as the driver
tries to reset the GPU [1]. My patch does not work for Exynos 5422 either
(but in a different way: GPU locks up in a few seconds).
So the first patch seems to be wrong and I'll drop it.
Best regards,
Alexey
[1]
[ 73.061941] panfrost 11800000.gpu: AS_ACTIVE bit stuck
[ 73.065671] 8<--- cut here ---
[ 73.067015] Power domain G3D disable failed
[ 73.068637] Unhandled fault: external abort on non-linefetch (0x1008) at 0xf09d0020
[ 73.080424] pgd = f347788b
[ 73.083108] [f09d0020] *pgd=414eb811, *pte=11800653, *ppte=11800453
[ 73.089352] Internal error: : 1008 [#1] PREEMPT SMP ARM
[ 73.094549] Modules linked in: xfrm_user l2tp_ppp l2tp_netlink l2tp_core ip6_udp_tunnel udp_tunnel pppox ppp_generic slhc rfkill loop zstd input_leds panfrost lzo_rle gpu_sched evdev uio_pdrv_genirq uio exynos_gpiomem zram sch_fq_codel ip_tables ipv6 btrfs blake2b_generic xor xor_neon zstd_compress lzo_compress zlib_deflate raid6_pq libcrc32c usbhid gpio_keys
[ 73.126264] CPU: 3 PID: 130 Comm: kworker/u16:3 Not tainted 5.15.8-00057-g5ecb31848317 #1
[ 73.134407] Hardware name: Hardkernel ODROID-XU4
[ 73.139001] Workqueue: panfrost-reset panfrost_reset_work [panfrost]
[ 73.145325] PC is at panfrost_gpu_soft_reset+0xa0/0x104 [panfrost]
[ 73.151477] LR is at schedule_hrtimeout_range_clock+0x114/0x240
[ 73.157370] pc : [<bf2c092c>] lr : [<c0bca590>] psr: 600e0013
[ 73.163609] sp : c16f3e88 ip : 00004710 fp : c16f3ea4
[ 73.168809] r10: c1255220 r9 : c5982a88 r8 : c5982a88
[ 73.174007] r7 : c5982a7c r6 : 00000011 r5 : 0364a751 r4 : c5982840
[ 73.180506] r3 : f09d0000 r2 : 00000000 r1 : 00000000 r0 : 00000000
[ 73.187006] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
[ 73.194112] Control: 10c5387d Table: 45d5c06a DAC: 00000051
[ 73.199830] Register r0 information: NULL pointer
[ 73.204509] Register r1 information: NULL pointer
[ 73.209189] Register r2 information: NULL pointer
[ 73.213868] Register r3 information: 0-page vmalloc region starting at 0xf09d0000 allocated at __devm_ioremap_resource+0x170/0x1e8
[ 73.225566] Register r4 information: slab kmalloc-1k start c5982800 pointer offset 64 size 1024
[ 73.234232] Register r5 information: non-paged memory
[ 73.239258] Register r6 information: non-paged memory
[ 73.244284] Register r7 information: slab kmalloc-1k start c5982800 pointer offset 636 size 1024
[ 73.253036] Register r8 information: slab kmalloc-1k start c5982800 pointer offset 648 size 1024
[ 73.261788] Register r9 information: slab kmalloc-1k start c5982800 pointer offset 648 size 1024
[ 73.270540] Register r10 information: non-slab/vmalloc memory
[ 73.276259] Register r11 information: non-slab/vmalloc memory
[ 73.281978] Register r12 information: non-paged memory
[ 73.287091] Process kworker/u16:3 (pid: 130, stack limit = 0x6da5c776)
[ 73.293591] Stack: (0xc16f3e88 to 0xc16f4000)
[ 73.297926] 3e80: c5982840 00000000 00000000 c5982a7c c16f3ebc c16f3ea8
[ 73.306072] 3ea0: bf2bf65c bf2c0898 c5982840 00000000 c16f3ef4 c16f3ec0 bf2c278c bf2bf64c
[ 73.314218] 3ec0: c16f3ef4 c16f3ed0 c015cf28 c5982aa4 c44ba900 c1410a00 c1e9c400 00000000
[ 73.322363] 3ee0: c1e9c405 c1255220 c16f3f04 c16f3ef8 bf2c29cc bf2c2538 c16f3f44 c16f3f08
[ 73.330509] 3f00: c014b3ac bf2c29b8 c1410a00 c1410a00 c1410a00 c1410a00 c1410a00 c44ba900
[ 73.338654] 3f20: c1410a00 c44ba918 c1410a18 c1103d00 00000088 c1410a00 c16f3f7c c16f3f48
[ 73.346800] 3f40: c014bc18 c014b1c0 c127b138 c16f2000 c0151004 c44bb880 c3747500 c014bbb4
[ 73.354945] 3f60: c44ba900 c16f2000 c35e5e8c c3747520 c16f3fac c16f3f80 c0152ee8 c014bbc0
[ 73.363091] 3f80: 00000000 c44bb880 c0152d7c 00000000 00000000 00000000 00000000 00000000
[ 73.371236] 3fa0: 00000000 c16f3fb0 c01000fc c0152d88 00000000 00000000 00000000 00000000
[ 73.379381] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 73.387526] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
[ 73.395669] Backtrace:
[ 73.398096] [<bf2c088c>] (panfrost_gpu_soft_reset [panfrost]) from [<bf2bf65c>] (panfrost_device_reset+0x1c/0x38 [panfrost])
[ 73.409278] r7:c5982a7c r6:00000000 r5:00000000 r4:c5982840
[ 73.414906] [<bf2bf640>] (panfrost_device_reset [panfrost]) from [<bf2c278c>] (panfrost_reset+0x260/0x378 [panfrost])
[ 73.425479] r5:00000000 r4:c5982840
[ 73.429031] [<bf2c252c>] (panfrost_reset [panfrost]) from [<bf2c29cc>] (panfrost_reset_work+0x20/0x24 [panfrost])
[ 73.439260] r10:c1255220 r9:c1e9c405 r8:00000000 r7:c1e9c400 r6:c1410a00 r5:c44ba900
[ 73.447055] r4:c5982aa4
[ 73.449568] [<bf2c29ac>] (panfrost_reset_work [panfrost]) from [<c014b3ac>] (process_one_work+0x1f8/0x5c0)
[ 73.459186] [<c014b1b4>] (process_one_work) from [<c014bc18>] (worker_thread+0x64/0x580)
[ 73.467250] r10:c1410a00 r9:00000088 r8:c1103d00 r7:c1410a18 r6:c44ba918 r5:c1410a00
[ 73.475045] r4:c44ba900
[ 73.477557] [<c014bbb4>] (worker_thread) from [<c0152ee8>] (kthread+0x16c/0x1a0)
[ 73.484927] r10:c3747520 r9:c35e5e8c r8:c16f2000 r7:c44ba900 r6:c014bbb4 r5:c3747500
[ 73.492722] r4:c44bb880
[ 73.495234] [<c0152d7c>] (kthread) from [<c01000fc>] (ret_from_fork+0x14/0x38)
[ 73.502427] Exception stack(0xc16f3fb0 to 0xc16f3ff8)
[ 73.507454] 3fa0: 00000000 00000000 00000000 00000000
[ 73.515601] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 73.523746] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000
[ 73.530334] r10:00000000 r9:00000000 r8:00000000 r7:00000000 r6:00000000 r5:c0152d7c
[ 73.538129] r4:c44bb880 r3:00000000
[ 73.541685] Code: e3a0001a ba000010 eb64253c e594300c (e5933020)
[ 73.547753] ---[ end trace 1af2dce52aebcc96 ]---
However my patch does not work for Exynos
5422 either, so I'll drop the first patch.
>
> Thanks,
> Robin.
>
> > Signed-off-by: Alexey Sheplyakov <asheplyakov at basealt.ru>
> > Signed-off-by: Vadim V. Vlasov <vadim.vlasov at elpitech.ru>
> >
> > Cc: Rob Herring <robh at kernel.org>
> > Cc: Tomeu Vizoso <tomeu.vizoso at collabora.com>
> > Cc: Steven Price <steven.price at arm.com>
> > Cc: Alyssa Rosenzweig <alyssa.rosenzweig at collabora.com>
> > Cc: Vadim V. Vlasov <vadim.vlasov at elpitech.ru>
> > ---
> > drivers/gpu/drm/panfrost/panfrost_mmu.c | 3 ---
> > drivers/iommu/io-pgtable-arm.c | 16 ++++++++++++----
> > 2 files changed, 12 insertions(+), 7 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/panfrost/panfrost_mmu.c b/drivers/gpu/drm/panfrost/panfrost_mmu.c
> > index 39562f2d11a4..2f4f8a17bc82 100644
> > --- a/drivers/gpu/drm/panfrost/panfrost_mmu.c
> > +++ b/drivers/gpu/drm/panfrost/panfrost_mmu.c
> > @@ -133,9 +133,6 @@ static void panfrost_mmu_enable(struct panfrost_device *pfdev, struct panfrost_m
> > mmu_write(pfdev, AS_TRANSTAB_LO(as_nr), lower_32_bits(transtab));
> > mmu_write(pfdev, AS_TRANSTAB_HI(as_nr), upper_32_bits(transtab));
> > - /* Need to revisit mem attrs.
> > - * NC is the default, Mali driver is inner WT.
> > - */
> > mmu_write(pfdev, AS_MEMATTR_LO(as_nr), lower_32_bits(memattr));
> > mmu_write(pfdev, AS_MEMATTR_HI(as_nr), upper_32_bits(memattr));
> > diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
> > index dd9e47189d0d..15b39c337e20 100644
> > --- a/drivers/iommu/io-pgtable-arm.c
> > +++ b/drivers/iommu/io-pgtable-arm.c
> > @@ -122,13 +122,17 @@
> > #define ARM_LPAE_MAIR_ATTR_IDX_CACHE 1
> > #define ARM_LPAE_MAIR_ATTR_IDX_DEV 2
> > #define ARM_LPAE_MAIR_ATTR_IDX_INC_OCACHE 3
> > +#define ARM_LPAE_MAIR_ATTR_IDX_OUTER_WA 4
> > #define ARM_MALI_LPAE_TTBR_ADRMODE_TABLE (3u << 0)
> > #define ARM_MALI_LPAE_TTBR_READ_INNER BIT(2)
> > #define ARM_MALI_LPAE_TTBR_SHARE_OUTER BIT(4)
> > -#define ARM_MALI_LPAE_MEMATTR_IMP_DEF 0x88ULL
> > -#define ARM_MALI_LPAE_MEMATTR_WRITE_ALLOC 0x8DULL
> > +#define ARM_MALI_LPAE_MEMATTR_IMP_DEF 0x48ULL
> > +#define ARM_MALI_LPAE_MEMATTR_FORCE_CACHE_ALL 0x4FULL
> > +#define ARM_MALI_LPAE_MEMATTR_WRITE_ALLOC 0x4DULL
> > +#define ARM_MALI_LPAE_MEMATTR_OUTER_IMP_DEF 0x88ULL
> > +#define ARM_MALI_LPAE_MEMATTR_OUTER_WA 0x8DULL
> > #define APPLE_DART_PTE_PROT_NO_WRITE (1<<7)
> > #define APPLE_DART_PTE_PROT_NO_READ (1<<8)
> > @@ -1080,10 +1084,14 @@ arm_mali_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg, void *cookie)
> > cfg->arm_mali_lpae_cfg.memattr =
> > (ARM_MALI_LPAE_MEMATTR_IMP_DEF
> > << ARM_LPAE_MAIR_ATTR_SHIFT(ARM_LPAE_MAIR_ATTR_IDX_NC)) |
> > + (ARM_MALI_LPAE_MEMATTR_FORCE_CACHE_ALL
> > + << ARM_LPAE_MAIR_ATTR_SHIFT(ARM_LPAE_MAIR_ATTR_IDX_CACHE)) |
> > (ARM_MALI_LPAE_MEMATTR_WRITE_ALLOC
> > << ARM_LPAE_MAIR_ATTR_SHIFT(ARM_LPAE_MAIR_ATTR_IDX_CACHE)) |
> > - (ARM_MALI_LPAE_MEMATTR_IMP_DEF
> > - << ARM_LPAE_MAIR_ATTR_SHIFT(ARM_LPAE_MAIR_ATTR_IDX_DEV));
> > + (ARM_MALI_LPAE_MEMATTR_OUTER_IMP_DEF
> > + << ARM_LPAE_MAIR_ATTR_SHIFT(ARM_LPAE_MAIR_ATTR_IDX_DEV)) |
> > + (ARM_MALI_LPAE_MEMATTR_OUTER_WA
> > + << ARM_LPAE_MAIR_ATTR_SHIFT(ARM_LPAE_MAIR_ATTR_IDX_OUTER_WA));
> > data->pgd = __arm_lpae_alloc_pages(ARM_LPAE_PGD_SIZE(data), GFP_KERNEL,
> > cfg);
More information about the dri-devel
mailing list