[PATCH 1/1] [RFC] drm/ttm: Don't init dma32_zone on 64-bit systems

[Kuehling, Felix]

On 2019-02-21 12:34 p.m., Thomas Hellstrom wrote:
> On Thu, 2019-02-21 at 16:57 +0000, Kuehling, Felix wrote:
>> On 2019-02-21 2:59 a.m., Koenig, Christian wrote:
>>> On x86 with HIGHMEM there is no dma32 zone. Why do we need one on
>>>>> x86_64? Can we make x86_64 more like HIGHMEM instead?
>>>>>
>>>>> Regards,
>>>>>       Felix
>>>>>
>>>> IIRC with x86, the kernel zone is always smaller than any dma32
>>>> zone,
>>>> so we'd always exhaust the kernel zone before dma32 anyway.
>>>>
>>>> Not sure why we have dma32 on x86 without highmem, though. sounds
>>>> superflous but harmless.
>>> Well DMA32 denotes memory which is accessible by devices who can
>>> only do
>>> 32bit addressing. And IIRC we can actually do DMA32 to highmem
>>> since
>>> something like 2.4.*.
>>>
>>> Because of this it is actually irrelevant if you have highmem or
>>> not,
>>> what matters for DMA32 is if you have an IOMMU or not.
>> Are you saying we should have a dma32_zone even on x86 with HIGHMEM?
>>
>>
>>> So even on x86_64 you actually do need the DMA32 zone if you don't
>>> have
>>> an IOMMU which remaps all memory for devices which can't directly
>>> address it.
>> Why is DMA32 special in this way? For example AMD GFX8 GPUs support
>> 40-bit DMA. But we don't have a special zone for that.
> If you're running on a non-IOMMU system with physical memory addresses
>> 40 bits, and tell the DMA subsystem that you need to restrict to 40
> bits, it will probably start using bounce buffers for streaming DMA
> (which won't work with most graphics drivers), or for
> dma_alloc_coherent(), it will probably use memory from the DMA32 zone.

OK, then why is it not needed when CONFIG_HIGHMEM is defined?

I found that there is a CONFIG_ZONE_DMA32 parameter. Maybe we should use 
that to decide whether to account for the DMA32 zone in TTM. It is set 
on x86_64 and a number of other 64-bit architectures.


>> How common is it to have devices that need DMA32 on an x86_64 system?
> IIRC All devices using dma_alloc_coherent() allocate DMA32 memory
> unless they explicitly set the dma coherent mask to something larger.
> Like Christian says, if an IOMMU is present and enabled, the need for
> the DMA32 zone goes away. In theory at least.

Thanks. I read up a bit on DMA32 and memory zones in general. I found 
that there is a lowmem_reserve_ratio feature that protects against 
normal page allocations overflowing into lowmem zones. There is some 
documentation in Documentation/scsctl/vm.txt (search for 
lowmem_reserve_ratio). The protected amount of memory in each zone can 
be seen in /proc/zoneinfo.

With that, can we conclude that we don't need to count 
ttm_mem_global_alloc against the dma32 zone.

Thanks,
   Felix


>
> /Thomas
>
>
>> Regards,
>>     Felix
>>
>>
>>> Regards,
>>> Christian.
>>>
>>>> /Thomas
>>>>
>>>>