[PATCH] drm/[amdgpu|radeon]: fix memset on io mem

[Chen Li]

On Thu, 17 Dec 2020 22:16:59 +0800,
Christian König wrote:
> 
> Am 17.12.20 um 14:37 schrieb Chen Li:
> > On Thu, 17 Dec 2020 18:25:11 +0800,
> > Christian König wrote:
> >> Am 17.12.20 um 02:07 schrieb Chen Li:
> >>> On Wed, 16 Dec 2020 22:19:11 +0800,
> >>> Christian König wrote:
> >>>> Am 16.12.20 um 14:48 schrieb Chen Li:
> >>>>> On Wed, 16 Dec 2020 15:59:37 +0800,
> >>>>> Christian König wrote:
> >>>>>> [SNIP]
> >>>>> Hi, Christian. I'm not sure why this change is a hack here. I cannot see the problem and wll be grateful if you give more explainations.
> >>>> __memset is supposed to work on those addresses, otherwise you can't use the
> >>>> e8860 on your arm64 system.
> >>> If __memset is supposed to work on those adresses, why this commit(https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftorvalds%2Flinux%2Fcommit%2Fba0b2275a6781b2f3919d931d63329b5548f6d5f&data=04%7C01%7Cchristian.koenig%40amd.com%7Cfdb4ca3e05ad4ea4882408d8a2914fbc%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637438092297678363%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=88oAUlEhnsVNSqYIfXk%2B811oXYd18XPScVZ4ceAurNk%3D&reserved=0) is needed? (I also notice drm/radeon didn't take this change though) just out of curiosity.
> >> We generally accept those patches as cleanup in the kernel with the hope that we
> >> can find a way to work around the userspace restrictions.
> > What's the userspace restriction here? mmap device memory?
> 
> Yes, exactly that.
> 
> >> But when you also have this issue in userspace then there isn't much we can do
> >> for you.
> >> 
> >>>> Replacing the the direct write in the kernel with calls to writel() or
> >>>> memset_io() will fix that temporary, but you have a more general problem here.
> >>>    I cannot see what's the more general problem here :( u mean performance?
> >> No, not performance. See standards like OpenGL, Vulkan as well as VA-API and
> >> VDPAU require that you can mmap() device memory and execute memset/memcpy on the
> >> memory from userspace.
> >> 
> >> If your ARM base board can't do that for some then you can't use the hardware
> >> with that board.
> > Good to know, thanks! BTW, have you ever seen or heard boards like mine which cannot mmap device memory correctly from userspace correctly?
> 
> Unfortunately yes. We haven't been able to figure out what exactly goes wrong in
> those cases.

Ok. one more question: only e8860 or all radeon cards have this issue?
 
> >>>>>> For amdgpu I suggest that we allocate the UVD message in GTT instead of VRAM
> >>>>>> since we don't have the hardware restriction for that on the new generations.
> >>>>>> 
> >>>>> Thanks, I will try to dig into deeper. But what's the "hardware restriction" meaning here? I'm not familiar with video driver stack and amd gpu, sorry.
> >>>> On older hardware (AGP days) the buffer had to be in VRAM (MMIO) memory, but on
> >>>> modern system GTT (system memory) works as well.
> >>> IIUC, e8860 can use amdgpu(I use radeon now) beause its device id 6822 is in amdgpu's table. But I cannot tell whether e8860 has iommu, and I cannot find iommu from lspci, so graphics translation table may not work here?
> >> That is not related to IOMMU. IOMMU is a feature of the CPU/motherboard. This is
> >> implemented using GTT, e.g. the VM page tables inside the GPU.
> >> 
> >> And yes it should work I will prepare a patch for it.
> > I think you mean mmu :)
> 
> No, I really meant IOMMU.
> 
> > Refer to wikipedia: https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fen.wikipedia.org%2Fwiki%2FInput%25E2%2580%2593output_memory_management_unit%23:~:text%3DIn%2520computing%252C%2520an%2520input%25E2%2580%2593output%2Cbus%2520to%2520the%2520main%2520memory&data=04%7C01%7Cchristian.koenig%40amd.com%7Cfdb4ca3e05ad4ea4882408d8a2914fbc%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637438092297678363%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=t6NDi8didU7GFzaCSMFvdSTKA%2FmRZ1cgPCpY7lf7UKo%3D&reserved=0.
> > 
> >      In computing, an input–output memory management unit (IOMMU) is a memory management unit (MMU) that connects a direct-memory-access–capable (DMA-capable) I/O bus to the main memory. Like a traditional MMU, which translates CPU-visible virtual addresses to physical addresses, the IOMMU maps device-visible virtual addresses (also called device addresses or I/O addresses in this context) to physical addresses. Some units also provide memory protection from faulty or malicious devices.
> >      An example IOMMU is the graphics address remapping table (GART) used by AGP and PCI Express graphics cards on Intel Architecture and AMD computers.
> 
> Maybe somebody should clarify the wikipedia article a bit since this is to
> general and misleading.
> 
> The key difference is that today IOMMU usually refers to the MMU block in the
> PCIe root complex of the CPU.
> 
> > GART should be antoher abber of GTT(https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FGraphics_address_remapping_table&data=04%7C01%7Cchristian.koenig%40amd.com%7Cfdb4ca3e05ad4ea4882408d8a2914fbc%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637438092297678363%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=b58oodroRED7%2FOocJQIJ5l9x6Ro5p895EIcR%2F3vExB0%3D&reserved=0):
> > 
> >      The graphics address remapping table (GART),[1] also known as the graphics aperture remapping table,[2] or graphics translation table (GTT),[3] is an I/O memory management unit (IOMMU) used by Accelerated Graphics Port (AGP) and PCI Express (PCIe) graphics cards.
> 
> GART or GTT refers to the translation tables graphics hardware use to access
> system memory.
> 
> Something like 15 years ago we used the IOMMU functionality from AGP to
> implement that. But modern hardware (PCIe) uses some specialized hardware in the
> GPU for that.
> 
> Regards,
> Christian.
> 
> 
> 

Good to know, thanks! So modern GART/GTT is like tlb, and iommu is forcused on translating address and not manager the tlb.