[Intel-gfx] [PATCH v5 5/5] drm/tinydrm: Switch from CMA to shmem buffers

[Noralf Trønnes]

Den 29.10.2018 10.07, skrev Daniel Vetter:
> On Sun, Oct 28, 2018 at 09:46:43PM +0100, Noralf Trønnes wrote:
>> Den 28.10.2018 21.21, skrev David Lechner:
>>> On 10/26/2018 05:38 PM, Noralf Trønnes wrote:
>>>> Den 17.10.2018 15.04, skrev Noralf Trønnes:
>>>>> This move makes tinydrm useful for more drivers. tinydrm doesn't need
>>>>> continuous memory, but at the time it was convenient to use the CMA
>>>>> library. The spi core can do dma on is_vmalloc() addresses making this
>>>>> possible.
>>>>>
>>>>> Cc: David Lechner <david at lechnology.com>
>>>>> Signed-off-by: Noralf Trønnes <noralf at tronnes.org>
>>>>> Acked-by: David Lechner <david at lechnology.com>
>>>>> Tested-by: David Lechner <david at lechnology.com>
>>>>> ---
>>>> David,
>>>> FYI This series is scratched.
>>>> See the shmem helper patch thread for details.
>>>>
>>> Yes, I saw that. Thank you.
>>>
>>> I don't suppose there is a way to configure the DMA controller to do
>>> the byte swapping?
>>>
>> Not that I know of.
>> "Proper" SPI hw can do 16-bit transfers and for them there's no problem.
>> The DMA capable SPI block on the Pi can only do 8-bit, hence the swapping.
>>
>> But that wasn't the problem, the byteswapping actually papered over the
>> problem. I'm no -mm expert so I don't know why the problem onyl showed
>> up when using the virtual address of the buffer used by fbcon and not on
>> mmap'ed fbdev as Daniel suggested would happen.
> Hm, I missed that detail. This sounds like one of the mappings ended up
> being write-combining (which avoids all the issues with dirty cpu cache
> lines), while the broken one was not.
>
> Or we ended up with a flush somewhere by accident.
>
>> Either way shmem not being coherent is a problem on the Pi, even though
>> I expected the DMA streaming API called by the spi core to flush "things".
> It should do that for you. At least if it's using dma_map/unmap_sg and
> friends.

I just watched a talk by the i2c maintainer about his challenges with
supporting DMA buffers and DMA only host controllers:

   DMA Safety in Buffers for Linux Kernel Device Drivers - Wolfram Sang
   https://www.youtube.com/watch?v=JDwaMClvV-s

In passing he mentions this from Documentation/spi/spi-summary:

   - I/O buffers use the usual Linux rules, and must be DMA-safe.
     You'd normally allocate them from the heap or free page pool.

So how can the SPI core ensure DMA safety for its DMA mapping of the
underlying pages of virtual addresses? This is what I relied on when
doing the shmem helper.

Doing further digging I came across these:

Vignesh R says:
https://www.spinics.net/lists/kernel/msg2687381.html

   SPI core does try to DMA into underlying physical pages of vmalloc'd
   buffer. But this has two problems:

   1. Does not work well with VIVT caches[1].
   2. On ARM LPAE systems, vmalloc'd buffers can be from highmem region
   that are not addressable using 32 bit addresses and is backed by LPAE.
   So, a 32 bit DMA cannot access these buffers.

   Both these issues lead to random crashes and errors with UBIFS and JFFS2
   flash file systems which this patch series tries to address using bounce
   buffer

   [1] https://patchwork.kernel.org/patch/9579553/


Russel King provides some more info:
https://patchwork.kernel.org/patch/9579553/#20139929

   SPI is rather another special case - rather than SPI following the
   established mechanism of passing data references via scatterlists or
   similar, it also passes them via virtual addresses, which means SPI
   can directly access the vmalloc area when performing PIO.  This
   really makes the problem more complex, because it means that if you
   do have a SPI driver that does that, it's going to be reading/writing
   direct from vmalloc space.

   That's not a problem as long as the data is only accessed via vmalloc
   space, but it will definitely go totally wrong if the data is
   subsequently mapped into userspace.


So even if I could make the shmem helper work on the Raspberry Pi with
tinydrm and SPI, it probably wouldn't work on all systems.


I also learned that using devm_kmalloc, which tinydrm does for a tx buffer,
is not DMA safe because it prepends a devres struct to the buffer making
the buffer itself not cache line aligned.

[Question] devm_kmalloc() for DMA ?
https://linux-arm-kernel.infradead.narkive.com/vyJqy0RQ/question-devm-kmalloc-for-dma

I'll turn this into a plain kmalloc when I return to my work on supporting
device unplug which will deal with device resource lifetime issues.

Noralf.