etnaviv: mmu issue after end of address space reached?

[Lucas Stach]

Hi Wladimir,

Am Samstag, den 10.12.2016, 18:05 +0100 schrieb Wladimir J. van der
Laan:
> > So the MMU fault is somehow specific to what I'm doing. Interesting.
> 
> I think I found the issue: the MMU "flush and sync" is not good enough in some
> cases.
> 
> What the Vivante kernel driver does, for MMUv2, after mapping some kinds of
> buffer objects (apparently those tagged INDEX and VERTEX, this includes shader
> code and CL buffers) is 
> 
> - Send MMU flush command (like we do)
> - Add a notify event "resume" (they hardwire event 29 for this)
> - Add END command the command buffer so that the FE stops
> - Remember where to continue
> 
> Then in the interrupt handler:
> 
> - If the "resume" notify event comes in
>     - Wait for FE to be idle
>     - Restart the FE to the remembered position
> 
> This is implemented in "pause" here http://git.freescale.com/git/cgit.cgi/imx/linux-2.6-imx.git/tree/drivers/mxc/gpu-viv/hal/kernel/gc_hal_kernel_command.c?id=77f61547834c4f127b44b13e43c59133a35880dc#n395
> gcvPAGE_TABLE_DIRTY_BIT_FE is set here: http://git.freescale.com/git/cgit.cgi/imx/linux-2.6-imx.git/tree/drivers/mxc/gpu-viv/hal/kernel/gc_hal_kernel_mmu.c?id=77f61547834c4f127b44b13e43c59133a35880dc#n2176
> endAfterFlushMmuCache is set here: http://git.freescale.com/git/cgit.cgi/imx/linux-2.6-imx.git/tree/drivers/mxc/gpu-viv/hal/kernel/arch/gc_hal_kernel_hardware.c?id=77f61547834c4f127b44b13e43c59133a35880dc#n1259
> The interrupt notification is handled here: http://git.freescale.com/git/cgit.cgi/imx/linux-2.6-imx.git/tree/drivers/mxc/gpu-viv/hal/kernel/gc_hal_kernel_event.c?id=77f61547834c4f127b44b13e43c59133a35880dc#n2224
> 
> I hacked this into the DRM driver and have been running my test for quite some time,
> bumping against the tail end of the address range many times, without any MMU faults.
> 
> My proposal is to add a bo flag for buffers that need this kind of "hard" MMU
> reset (this is not all of them, e.g. textures don't), and if their iova mapping
> requires a MMU flush, do the above stop-and-start ritual (in case of MMUv2).

I'm aware of what the Vivante driver does. Unfortunately we would
basically need to flush the MMU before each user command stream, as we
continuously map new command buffers into the IOVA, which would be
crippling for performance. Vivante gets around this by setting up a 1:1
virt:phys mapping by default.

The current etnaviv code gets around this stop->irq->start dance by
spacing out the command streams, which seems to be enough to get around
the FE MMU flush failure. This may not work correctly at the end of the
address range. I'll take a look at this.

Blindly implementing the Vivante way does not seem like the correct
approach to me.

Regards,
Lucas