[PATCH 1/1] [RFC] drm/ttm: Don't init dma32_zone on 64-bit systems

[Koenig, Christian]

Am 21.02.19 um 22:02 schrieb Thomas Hellstrom:
> Hi,
>
> On Thu, 2019-02-21 at 20:24 +0000, Kuehling, Felix wrote:
>> On 2019-02-21 12:34 p.m., Thomas Hellstrom wrote:
>>> On Thu, 2019-02-21 at 16:57 +0000, Kuehling, Felix wrote:
>>>> On 2019-02-21 2:59 a.m., Koenig, Christian wrote:
>>>>> On x86 with HIGHMEM there is no dma32 zone. Why do we need one
>>>>> on
>>>>>>> x86_64? Can we make x86_64 more like HIGHMEM instead?
>>>>>>>
>>>>>>> Regards,
>>>>>>>        Felix
>>>>>>>
>>>>>> IIRC with x86, the kernel zone is always smaller than any
>>>>>> dma32
>>>>>> zone,
>>>>>> so we'd always exhaust the kernel zone before dma32 anyway.
>>>>>>
>>>>>> Not sure why we have dma32 on x86 without highmem, though.
>>>>>> sounds
>>>>>> superflous but harmless.
>>>>> Well DMA32 denotes memory which is accessible by devices who
>>>>> can
>>>>> only do
>>>>> 32bit addressing. And IIRC we can actually do DMA32 to highmem
>>>>> since
>>>>> something like 2.4.*.
>>>>>
>>>>> Because of this it is actually irrelevant if you have highmem
>>>>> or
>>>>> not,
>>>>> what matters for DMA32 is if you have an IOMMU or not.
>>>> Are you saying we should have a dma32_zone even on x86 with
>>>> HIGHMEM?
>>>>
>>>>
>>>>> So even on x86_64 you actually do need the DMA32 zone if you
>>>>> don't
>>>>> have
>>>>> an IOMMU which remaps all memory for devices which can't
>>>>> directly
>>>>> address it.
>>>> Why is DMA32 special in this way? For example AMD GFX8 GPUs
>>>> support
>>>> 40-bit DMA. But we don't have a special zone for that.
>>> If you're running on a non-IOMMU system with physical memory
>>> addresses
>>>> 40 bits, and tell the DMA subsystem that you need to restrict to
>>>> 40
>>> bits, it will probably start using bounce buffers for streaming DMA
>>> (which won't work with most graphics drivers), or for
>>> dma_alloc_coherent(), it will probably use memory from the DMA32
>>> zone.
>> OK, then why is it not needed when CONFIG_HIGHMEM is defined?
>>
>> I found that there is a CONFIG_ZONE_DMA32 parameter. Maybe we should
>> use
>> that to decide whether to account for the DMA32 zone in TTM. It is
>> set
>> on x86_64 and a number of other 64-bit architectures.
>>
>>
>>>> How common is it to have devices that need DMA32 on an x86_64
>>>> system?
>>> IIRC All devices using dma_alloc_coherent() allocate DMA32 memory
>>> unless they explicitly set the dma coherent mask to something
>>> larger.
>>> Like Christian says, if an IOMMU is present and enabled, the need
>>> for
>>> the DMA32 zone goes away. In theory at least.
>> Thanks. I read up a bit on DMA32 and memory zones in general. I
>> found
>> that there is a lowmem_reserve_ratio feature that protects against
>> normal page allocations overflowing into lowmem zones. There is some
>> documentation in Documentation/scsctl/vm.txt (search for
>> lowmem_reserve_ratio). The protected amount of memory in each zone
>> can
>> be seen in /proc/zoneinfo.
>>
>> With that, can we conclude that we don't need to count
>> ttm_mem_global_alloc against the dma32 zone.
> Yes, it indeed looks like that.
> But then I would suggest removing the DMA32 zone entirely.

We still need it for the pages we allocate, but we should just stop 
accounting all the housekeeping to it.

To answer Felix earlier question we really still need DMA32 without 
IOMMU even on a 64bit system. The problem is simply that you have tons 
of PCIe hardware which can do only 32bit addressing, for example the 
audio codecs even on our newest Vega chips.

Regards,
Christian.

>
> /Thomas
>
>
>> Thanks,
>>     Felix
>>
>>
>>> /Thomas
>>>
>>>
>>>> Regards,
>>>>      Felix
>>>>
>>>>
>>>>> Regards,
>>>>> Christian.
>>>>>
>>>>>> /Thomas
>>>>>>
>>>>>>