[PATCH 1/1] [RFC] drm/ttm: Don't init dma32_zone on 64-bit systems

[Thomas Hellstrom]

Hi,

On Thu, 2019-02-21 at 20:24 +0000, Kuehling, Felix wrote:
> On 2019-02-21 12:34 p.m., Thomas Hellstrom wrote:
> > On Thu, 2019-02-21 at 16:57 +0000, Kuehling, Felix wrote:
> > > On 2019-02-21 2:59 a.m., Koenig, Christian wrote:
> > > > On x86 with HIGHMEM there is no dma32 zone. Why do we need one
> > > > on
> > > > > > x86_64? Can we make x86_64 more like HIGHMEM instead?
> > > > > > 
> > > > > > Regards,
> > > > > >       Felix
> > > > > > 
> > > > > IIRC with x86, the kernel zone is always smaller than any
> > > > > dma32
> > > > > zone,
> > > > > so we'd always exhaust the kernel zone before dma32 anyway.
> > > > > 
> > > > > Not sure why we have dma32 on x86 without highmem, though.
> > > > > sounds
> > > > > superflous but harmless.
> > > > Well DMA32 denotes memory which is accessible by devices who
> > > > can
> > > > only do
> > > > 32bit addressing. And IIRC we can actually do DMA32 to highmem
> > > > since
> > > > something like 2.4.*.
> > > > 
> > > > Because of this it is actually irrelevant if you have highmem
> > > > or
> > > > not,
> > > > what matters for DMA32 is if you have an IOMMU or not.
> > > Are you saying we should have a dma32_zone even on x86 with
> > > HIGHMEM?
> > > 
> > > 
> > > > So even on x86_64 you actually do need the DMA32 zone if you
> > > > don't
> > > > have
> > > > an IOMMU which remaps all memory for devices which can't
> > > > directly
> > > > address it.
> > > Why is DMA32 special in this way? For example AMD GFX8 GPUs
> > > support
> > > 40-bit DMA. But we don't have a special zone for that.
> > If you're running on a non-IOMMU system with physical memory
> > addresses
> > > 40 bits, and tell the DMA subsystem that you need to restrict to
> > > 40
> > bits, it will probably start using bounce buffers for streaming DMA
> > (which won't work with most graphics drivers), or for
> > dma_alloc_coherent(), it will probably use memory from the DMA32
> > zone.
> 
> OK, then why is it not needed when CONFIG_HIGHMEM is defined?
> 
> I found that there is a CONFIG_ZONE_DMA32 parameter. Maybe we should
> use 
> that to decide whether to account for the DMA32 zone in TTM. It is
> set 
> on x86_64 and a number of other 64-bit architectures.
> 
> 
> > > How common is it to have devices that need DMA32 on an x86_64
> > > system?
> > IIRC All devices using dma_alloc_coherent() allocate DMA32 memory
> > unless they explicitly set the dma coherent mask to something
> > larger.
> > Like Christian says, if an IOMMU is present and enabled, the need
> > for
> > the DMA32 zone goes away. In theory at least.
> 
> Thanks. I read up a bit on DMA32 and memory zones in general. I
> found 
> that there is a lowmem_reserve_ratio feature that protects against 
> normal page allocations overflowing into lowmem zones. There is some 
> documentation in Documentation/scsctl/vm.txt (search for 
> lowmem_reserve_ratio). The protected amount of memory in each zone
> can 
> be seen in /proc/zoneinfo.
> 
> With that, can we conclude that we don't need to count 
> ttm_mem_global_alloc against the dma32 zone.

Yes, it indeed looks like that.
But then I would suggest removing the DMA32 zone entirely.

/Thomas


> 
> Thanks,
>    Felix
> 
> 
> > /Thomas
> > 
> > 
> > > Regards,
> > >     Felix
> > > 
> > > 
> > > > Regards,
> > > > Christian.
> > > > 
> > > > > /Thomas
> > > > > 
> > > > >