[PATCH 1/1] [RFC] drm/ttm: Don't init dma32_zone on 64-bit systems

[Thomas Hellstrom]

On Fri, 2019-02-22 at 07:10 +0000, Koenig, Christian wrote:
> Am 21.02.19 um 22:02 schrieb Thomas Hellstrom:
> > Hi,
> > 
> > On Thu, 2019-02-21 at 20:24 +0000, Kuehling, Felix wrote:
> > > On 2019-02-21 12:34 p.m., Thomas Hellstrom wrote:
> > > > On Thu, 2019-02-21 at 16:57 +0000, Kuehling, Felix wrote:
> > > > > On 2019-02-21 2:59 a.m., Koenig, Christian wrote:
> > > > > > On x86 with HIGHMEM there is no dma32 zone. Why do we need
> > > > > > one
> > > > > > on
> > > > > > > > x86_64? Can we make x86_64 more like HIGHMEM instead?
> > > > > > > > 
> > > > > > > > Regards,
> > > > > > > >        Felix
> > > > > > > > 
> > > > > > > IIRC with x86, the kernel zone is always smaller than any
> > > > > > > dma32
> > > > > > > zone,
> > > > > > > so we'd always exhaust the kernel zone before dma32
> > > > > > > anyway.
> > > > > > > 
> > > > > > > Not sure why we have dma32 on x86 without highmem,
> > > > > > > though.
> > > > > > > sounds
> > > > > > > superflous but harmless.
> > > > > > Well DMA32 denotes memory which is accessible by devices
> > > > > > who
> > > > > > can
> > > > > > only do
> > > > > > 32bit addressing. And IIRC we can actually do DMA32 to
> > > > > > highmem
> > > > > > since
> > > > > > something like 2.4.*.
> > > > > > 
> > > > > > Because of this it is actually irrelevant if you have
> > > > > > highmem
> > > > > > or
> > > > > > not,
> > > > > > what matters for DMA32 is if you have an IOMMU or not.
> > > > > Are you saying we should have a dma32_zone even on x86 with
> > > > > HIGHMEM?
> > > > > 
> > > > > 
> > > > > > So even on x86_64 you actually do need the DMA32 zone if
> > > > > > you
> > > > > > don't
> > > > > > have
> > > > > > an IOMMU which remaps all memory for devices which can't
> > > > > > directly
> > > > > > address it.
> > > > > Why is DMA32 special in this way? For example AMD GFX8 GPUs
> > > > > support
> > > > > 40-bit DMA. But we don't have a special zone for that.
> > > > If you're running on a non-IOMMU system with physical memory
> > > > addresses
> > > > > 40 bits, and tell the DMA subsystem that you need to restrict
> > > > > to
> > > > > 40
> > > > bits, it will probably start using bounce buffers for streaming
> > > > DMA
> > > > (which won't work with most graphics drivers), or for
> > > > dma_alloc_coherent(), it will probably use memory from the
> > > > DMA32
> > > > zone.
> > > OK, then why is it not needed when CONFIG_HIGHMEM is defined?
> > > 
> > > I found that there is a CONFIG_ZONE_DMA32 parameter. Maybe we
> > > should
> > > use
> > > that to decide whether to account for the DMA32 zone in TTM. It
> > > is
> > > set
> > > on x86_64 and a number of other 64-bit architectures.
> > > 
> > > 
> > > > > How common is it to have devices that need DMA32 on an x86_64
> > > > > system?
> > > > IIRC All devices using dma_alloc_coherent() allocate DMA32
> > > > memory
> > > > unless they explicitly set the dma coherent mask to something
> > > > larger.
> > > > Like Christian says, if an IOMMU is present and enabled, the
> > > > need
> > > > for
> > > > the DMA32 zone goes away. In theory at least.
> > > Thanks. I read up a bit on DMA32 and memory zones in general. I
> > > found
> > > that there is a lowmem_reserve_ratio feature that protects
> > > against
> > > normal page allocations overflowing into lowmem zones. There is
> > > some
> > > documentation in Documentation/scsctl/vm.txt (search for
> > > lowmem_reserve_ratio). The protected amount of memory in each
> > > zone
> > > can
> > > be seen in /proc/zoneinfo.
> > > 
> > > With that, can we conclude that we don't need to count
> > > ttm_mem_global_alloc against the dma32 zone.
> > Yes, it indeed looks like that.
> > But then I would suggest removing the DMA32 zone entirely.
> 
> We still need it for the pages we allocate, but we should just stop 
> accounting all the housekeeping to it.

Why is that? Can't we just account all pages in the kernel zone, and
leave it up to the kernel to make sure there are still DMA32 pages
left?

/Thomas