[PATCH] drm/[amdgpu|radeon]: fix memset on io mem

Thu Dec 17 14:16:59 UTC 2020

Am 17.12.20 um 14:37 schrieb Chen Li:
> On Thu, 17 Dec 2020 18:25:11 +0800,
> Christian König wrote:
>> Am 17.12.20 um 02:07 schrieb Chen Li:
>>> On Wed, 16 Dec 2020 22:19:11 +0800,
>>> Christian König wrote:
>>>> Am 16.12.20 um 14:48 schrieb Chen Li:
>>>>> On Wed, 16 Dec 2020 15:59:37 +0800,
>>>>> Christian König wrote:
>>>>>> [SNIP]
>>>>> Hi, Christian. I'm not sure why this change is a hack here. I cannot see the problem and wll be grateful if you give more explainations.
>>>> __memset is supposed to work on those addresses, otherwise you can't use the
>>>> e8860 on your arm64 system.
>>> If __memset is supposed to work on those adresses, why this commit(https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftorvalds%2Flinux%2Fcommit%2Fba0b2275a6781b2f3919d931d63329b5548f6d5f&data=04%7C01%7Cchristian.koenig%40amd.com%7Cfdb4ca3e05ad4ea4882408d8a2914fbc%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637438092297678363%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=88oAUlEhnsVNSqYIfXk%2B811oXYd18XPScVZ4ceAurNk%3D&reserved=0) is needed? (I also notice drm/radeon didn't take this change though) just out of curiosity.
>> We generally accept those patches as cleanup in the kernel with the hope that we
>> can find a way to work around the userspace restrictions.
> What's the userspace restriction here? mmap device memory?

Yes, exactly that.

>> But when you also have this issue in userspace then there isn't much we can do
>> for you.
>>
>>>> Replacing the the direct write in the kernel with calls to writel() or
>>>> memset_io() will fix that temporary, but you have a more general problem here.
>>>    I cannot see what's the more general problem here :( u mean performance?
>> No, not performance. See standards like OpenGL, Vulkan as well as VA-API and
>> VDPAU require that you can mmap() device memory and execute memset/memcpy on the
>> memory from userspace.
>>
>> If your ARM base board can't do that for some then you can't use the hardware
>> with that board.
> Good to know, thanks! BTW, have you ever seen or heard boards like mine which cannot mmap device memory correctly from userspace correctly?

Unfortunately yes. We haven't been able to figure out what exactly goes 
wrong in those cases.

>>>>>> For amdgpu I suggest that we allocate the UVD message in GTT instead of VRAM
>>>>>> since we don't have the hardware restriction for that on the new generations.
>>>>>>
>>>>> Thanks, I will try to dig into deeper. But what's the "hardware restriction" meaning here? I'm not familiar with video driver stack and amd gpu, sorry.
>>>> On older hardware (AGP days) the buffer had to be in VRAM (MMIO) memory, but on
>>>> modern system GTT (system memory) works as well.
>>> IIUC, e8860 can use amdgpu(I use radeon now) beause its device id 6822 is in amdgpu's table. But I cannot tell whether e8860 has iommu, and I cannot find iommu from lspci, so graphics translation table may not work here?
>> That is not related to IOMMU. IOMMU is a feature of the CPU/motherboard. This is
>> implemented using GTT, e.g. the VM page tables inside the GPU.
>>
>> And yes it should work I will prepare a patch for it.
> I think you mean mmu :)

No, I really meant IOMMU.

> Refer to wikipedia: https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fen.wikipedia.org%2Fwiki%2FInput%25E2%2580%2593output_memory_management_unit%23:~:text%3DIn%2520computing%252C%2520an%2520input%25E2%2580%2593output%2Cbus%2520to%2520the%2520main%2520memory&data=04%7C01%7Cchristian.koenig%40amd.com%7Cfdb4ca3e05ad4ea4882408d8a2914fbc%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637438092297678363%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=t6NDi8didU7GFzaCSMFvdSTKA%2FmRZ1cgPCpY7lf7UKo%3D&reserved=0.
>
>      In computing, an input–output memory management unit (IOMMU) is a memory management unit (MMU) that connects a direct-memory-access–capable (DMA-capable) I/O bus to the main memory. Like a traditional MMU, which translates CPU-visible virtual addresses to physical addresses, the IOMMU maps device-visible virtual addresses (also called device addresses or I/O addresses in this context) to physical addresses. Some units also provide memory protection from faulty or malicious devices.
>      An example IOMMU is the graphics address remapping table (GART) used by AGP and PCI Express graphics cards on Intel Architecture and AMD computers.

Maybe somebody should clarify the wikipedia article a bit since this is 
to general and misleading.

The key difference is that today IOMMU usually refers to the MMU block 
in the PCIe root complex of the CPU.

> GART should be antoher abber of GTT(https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FGraphics_address_remapping_table&data=04%7C01%7Cchristian.koenig%40amd.com%7Cfdb4ca3e05ad4ea4882408d8a2914fbc%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637438092297678363%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=b58oodroRED7%2FOocJQIJ5l9x6Ro5p895EIcR%2F3vExB0%3D&reserved=0):
>
>      The graphics address remapping table (GART),[1] also known as the graphics aperture remapping table,[2] or graphics translation table (GTT),[3] is an I/O memory management unit (IOMMU) used by Accelerated Graphics Port (AGP) and PCI Express (PCIe) graphics cards.

GART or GTT refers to the translation tables graphics hardware use to 
access system memory.

Something like 15 years ago we used the IOMMU functionality from AGP to 
implement that. But modern hardware (PCIe) uses some specialized 
hardware in the GPU for that.

Regards,
Christian.