Use of pci_map_page in nouveau, radeon TTM.

[Konrad Rzeszutek Wilk]

On Tue, Oct 01, 2013 at 12:16:16PM +0200, Thomas Hellstrom wrote:
> Jerome, Konrad
> 
> Forgive an ignorant question, but it appears like both Nouveau and
> Radeon may use pci_map_page() when populating TTMs on
> pages obtained using the ordinary (not DMA pool). These pages will,
> if I understand things correctly, not be pages allocated with
> DMA_ALLOC_COHERENT.

Not always. That depends if the SWIOTLB buffer has been enabled.
Which happens if you have Calgary IOMMU, AMD GART and if you
run under Xen.
> 
> From what I understand, at least for the corresponding
> dma_map_page() it's illegal for the CPU to access these pages
> without calling
> dma_sync_xx_for_cpu(). And before the device is allowed to access
> them again, you need to call dma_sync_xx_for_device().

Correct.

> So mapping for PCI really invalidates the TTM interleaved CPU /
> device access model.

Unless you use the TTM DMA one which allocates them from the
coherent pool - in which case they are already mapped.

Granted the part of using DMA export/import API is not finished
(so moving from TTM pool to a V4L for example) and it will blow
up with the right mix.
> 
> Or did I miss something here?

That is it. But for most of the use cases the drivers have been
able to skirt this restriction b/c the pci_map_page/pci_unmap_page
setup a DMA mapping that is static (until the pci_unmap_page) and
on x86 the memory is coherent. So the map is good irregardless
of the PCI devices. Naturally if you have multitple IOMMUs per bridge
this all falls apart :-(

This all falls flat also with non-coherent memory and I believe
that is what some of the PA-RISC folks are hitting their heads
against.

And probably also on ARM once they start using these chipsets.

> 
> Thanks,
> Thomas