On Thu, Feb 04, 2021 at 11:49:23AM +0000, Robin Murphy wrote:

On 2021-02-04 07:29, Christoph Hellwig wrote:

...

On Wed, Feb 03, 2021 at 03:37:05PM -0800, Dongli Zhang wrote:

...

This patch converts several swiotlb related variables to arrays, in order to maintain stat/status for different swiotlb buffers. Here are variables involved:

• io_tlb_start and io_tlb_end
• io_tlb_nslabs and io_tlb_used
• io_tlb_list
• io_tlb_index
• max_segment
• io_tlb_orig_addr
• no_iotlb_memory

There is no functional change and this is to prepare to enable 64-bit swiotlb.

Claire Chang (on Cc) already posted a patch like this a month ago, which looks much better because it actually uses a struct instead of all the random variables.

Indeed, I skimmed the cover letter and immediately thought that this whole thing is just the restricted DMA pool concept[1] again, only from a slightly different angle.

Kind of. Let me lay out how some of these pieces are right now:

+-----------------------+ +----------------------+ | | | | | | | | | a)Xen-SWIOTLB | | b)SWIOTLB (for !Xen) | | | | | +-----------XX----------+ +-------X--------------+ XXXX XXXXXXXXX XXXX XX XXX X XX XXXX +----------XX-----------+ | | | | | c) SWIOTLB generic | | | +-----------------------+

Dongli's patches modify the SWIOTLB generic c), and Xen-SWIOTLB a) parts.

Also see the IOMMU_INIT logic which lays this a bit more deepth (for example how to enable SWIOTLB on AMD boxes, or IBM with Calgary IOMMU, etc - see iommu_table.h).

Furtheremore it lays the groundwork to allocate AMD SEV SWIOTLB buffers later after boot (so that you can stich different pools together). All the bits are kind of inside of the SWIOTLB code. And also it changes the Xen-SWIOTLB to do something similar.

The mempool did it similarly by taking the internal parts (aka the various io_tlb) of SWIOTLB and exposing them out and having other code:

+-----------------------+ +----------------------+ | | | | | | | | | a)Xen-SWIOTLB | | b)SWIOTLB (for !Xen) | | | | | +-----------XX----------+ +-------X--------------+ XXXX XXXXXXXXX XXXX XX XXX X XX XXXX +----------XX-----------+ +------------------+ | | | Device tree | | +<--------+ enabling SWIOTLB | |c) SWIOTLB generic | | | | | | mempool | +-----------------------+ +------------------+

What I was suggesting to Clarie to follow Xen model, that is do something like this:

+-----------------------+ +----------------------+ +--------------------+ | | | | | | | | | | | | | a)Xen-SWIOTLB | | b)SWIOTLB (for !Xen) | | e) DT-SWIOTLB | | | | | | | +-----------XX----------+ +-------X--------------+ +----XX-X------------+ XXXX XXXXXXXXX XXX X X XX X XX XXXX XX XXX XXXXXXXX X XX XXXXXXXXXXXXX XXXXXXXX +----------XXX----------+ | | | | |c) SWIOTLB generic | | | +-----------------------+

so using the SWIOTLB generic parts, and then bolt on top of the device-tree logic, along with the mempool logic.

But Christopher has an interesting suggestion which is to squash the all the existing code (a, b, c) all together and pepper it with various jump-tables.

So:

-----------------------------+ | SWIOTLB: | | | | a) SWIOTLB (for non-Xen) | | b) Xen-SWIOTLB | | c) DT-SWIOTLB | | | | | -----------------------------+

with all the various bits (M2P/P2M for Xen, mempool for ARM, and normal allocation for BM) in one big file.