<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> <meta name="Generator" content="Microsoft Word 15 (filtered medium)"> <style></style> </head> <body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word"> <p style="font-family:Arial;font-size:11pt;color:#0078D7;margin:5pt;" align="Left"> [AMD Official Use Only - Internal Distribution Only]<br> </p> <br> <div> <div class="WordSection1"> <p class="MsoNormal">Hi Christian,<o:p></o:p></p> <p class="MsoNormal"><o:p> </o:p></p> <p class="MsoNormal">Can you explain why __iomem annotation is mandatory for amdgpu driver? If this is the case, we can’t switch to memremap. The only fix seems to me is add a #ifdef __x86_64__ to the ioremap_cache codes.<o:p></o:p></p> <p class="MsoNormal"><o:p> </o:p></p> <div> <p class="MsoNormal">Regards,<o:p></o:p></p> <p class="MsoNormal">Oak<o:p></o:p></p> </div> <p class="MsoNormal"><o:p> </o:p></p> <div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt"> <div> <div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in"> <p class="MsoNormal"><b>From:</b> Christian König <ckoenig.leichtzumerken@gmail.com> <br> <b>Sent:</b> Wednesday, March 3, 2021 5:46 AM<br> <b>To:</b> Zeng, Oak <Oak.Zeng@amd.com>; amd-gfx@lists.freedesktop.org; dri-devel@lists.freedesktop.org; Daniel Vetter <daniel@ffwll.ch>; Dave Airlie <airlied@redhat.com>; Thomas Hellström (Intel) <thomas_os@shipmail.org>; dan.j.williams@intel.com<br> <b>Cc:</b> kbuild-all@lists.01.org; Kuehling, Felix <Felix.Kuehling@amd.com>; Kasiviswanathan, Harish <Harish.Kasiviswanathan@amd.com>; Deucher, Alexander <Alexander.Deucher@amd.com>; Huang, JinHuiEric <JinHuiEric.Huang@amd.com>; Koenig, Christian <Christian.Koenig@amd.com><br> <b>Subject:</b> Re: [PATCH] drm/ttm: ioremap buffer according to TTM mem caching setting<o:p></o:p></p> </div> </div> <p class="MsoNormal"><o:p> </o:p></p> <p class="MsoNormal">Hi Oak,<br> <br> <br> <o:p></o:p></p> <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"> <pre>config: parisc-randconfig-r012-20210302 (attached as .config)<o:p></o:p></pre> </blockquote> <p class="MsoNormal" style="margin-bottom:12.0pt"><br> It's not the Intel driver build which fails here, but the build bot is just hosted by Intel.<br> <br> The problem is that the parisc architecture doesn't defines the ioremap_cache() function.<br> <br> I've looked at using memremap() instead of ioremap_cache(). The problem is that we do support architectures with the TTM as well as amndgpu code where the __iomem annotation is mandatory and correct.<br> <br> Regards,<br> Christian.<o:p></o:p></p> <div> <p class="MsoNormal">Am 02.03.21 um 23:45 schrieb Zeng, Oak:<o:p></o:p></p> </div> <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"> <p style="margin:5.0pt"><span style="font-family:"Arial",sans-serif;color:#0078D7">[AMD Official Use Only - Internal Distribution Only]<o:p></o:p></span></p> <p class="MsoNormal"><o:p> </o:p></p> <div> <p class="MsoNormal">Hi Daniel, Thomas, Dan,<o:p></o:p></p> <p class="MsoNormal"> <o:p></o:p></p> <p class="MsoNormal">Does below message mean the calling ioremap_cache failed intel’s driver build? I can see both ioremap_cache and ioremap_wc are defined in arch/x86/mm/ioremap.c – why ioremap_wc doesn’t break intel driver’s build?<o:p></o:p></p> <p class="MsoNormal"> <o:p></o:p></p> <p class="MsoNormal">Are we supposed to use memremap (offset, size, MEMREMAP_WB) to replace ioremap_cache? When I read here <a href="https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flwn.net%2FArticles%2F653585%2F&data=04%7C01%7COak.Zeng%40amd.com%7Cc047ecb316df47cde7ed08d8de3188d4%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637503651624296472%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ljCFrIfrYbb%2FXmKKS2TJ7dSQ7oCRNWoUhWS4gEBv%2FW4%3D&reserved=0"> https://lwn.net/Articles/653585/</a> I felt that ioremap_cache returns an address annotated with _iomem while memremap returns an address without __iomem annotation. In our use case, GPU memory is treated as UEFI SPM (specific purpose memory). I am not very sure whether memremap (thus no __iomem annotation) is the right thing to do. What I am sure is, we have tested ioremap_cache and it works fine on AMD system.<o:p></o:p></p> <p class="MsoNormal"> <o:p></o:p></p> <p class="MsoNormal">I will send out a test patch replacing ioremap_cache with ioremap_wc, to trigger Intel build robot to see whether it fails Intel build. I suppose it will not fail Intel build.<o:p></o:p></p> <p class="MsoNormal"> <o:p></o:p></p> <div> <p class="MsoNormal">Regards,<o:p></o:p></p> <p class="MsoNormal">Oak<o:p></o:p></p> </div> <p class="MsoNormal"> <o:p></o:p></p> <div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt"> <div> <div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in"> <p class="MsoNormal"><b>From:</b> Christian König <a href="mailto:ckoenig.leichtzumerken@gmail.com"> <ckoenig.leichtzumerken@gmail.com></a> <br> <b>Sent:</b> Tuesday, March 2, 2021 6:31 AM<br> <b>To:</b> <a href="mailto:amd-gfx@lists.freedesktop.org">amd-gfx@lists.freedesktop.org</a>; <a href="mailto:dri-devel@lists.freedesktop.org">dri-devel@lists.freedesktop.org</a>; Daniel Vetter <a href="mailto:daniel@ffwll.ch"><daniel@ffwll.ch></a>; Dave Airlie <a href="mailto:airlied@redhat.com"> <airlied@redhat.com></a>; Thomas Hellström (Intel) <a href="mailto:thomas_os@shipmail.org"> <thomas_os@shipmail.org></a><br> <b>Cc:</b> Zeng, Oak <a href="mailto:Oak.Zeng@amd.com"><Oak.Zeng@amd.com></a>; <a href="mailto:kbuild-all@lists.01.org"> kbuild-all@lists.01.org</a>; Kuehling, Felix <a href="mailto:Felix.Kuehling@amd.com"> <Felix.Kuehling@amd.com></a>; Kasiviswanathan, Harish <a href="mailto:Harish.Kasiviswanathan@amd.com"> <Harish.Kasiviswanathan@amd.com></a>; Deucher, Alexander <a href="mailto:Alexander.Deucher@amd.com"> <Alexander.Deucher@amd.com></a>; Huang, JinHuiEric <a href="mailto:JinHuiEric.Huang@amd.com"> <JinHuiEric.Huang@amd.com></a>; Koenig, Christian <a href="mailto:Christian.Koenig@amd.com"> <Christian.Koenig@amd.com></a><br> <b>Subject:</b> Re: [PATCH] drm/ttm: ioremap buffer according to TTM mem caching setting<o:p></o:p></p> </div> </div> <p class="MsoNormal"> <o:p></o:p></p> <p class="MsoNormal" style="margin-bottom:12.0pt">Hi guys,<br> <br> adding the usual suspects direct. Does anybody of hand know how to check if an architecture supports ioremap_cache()?<br> <br> For now we only need this on X86, but I would feel better if we don't use an #ifdef here.<br> <br> Regards,<br> Christian.<o:p></o:p></p> <div> <p class="MsoNormal">Am 02.03.21 um 05:12 schrieb kernel test robot:<o:p></o:p></p> </div> <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"> <pre>Hi Oak,<o:p></o:p></pre> <pre> <o:p></o:p></pre> <pre>Thank you for the patch! Yet something to improve:<o:p></o:p></pre> <pre> <o:p></o:p></pre> <pre>[auto build test ERROR on drm-intel/for-linux-next]<o:p></o:p></pre> <pre>[also build test ERROR on drm-tip/drm-tip linus/master v5.12-rc1 next-20210302]<o:p></o:p></pre> <pre>[cannot apply to tegra-drm/drm/tegra/for-next drm-exynos/exynos-drm-next drm/drm-next]<o:p></o:p></pre> <pre>[If your patch is applied to the wrong git tree, kindly drop us a note.<o:p></o:p></pre> <pre>And when submitting patch, we suggest to use '--base' as documented in<o:p></o:p></pre> <pre><a href="https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit-scm.com%2Fdocs%2Fgit-format-patch&data=04%7C01%7COak.Zeng%40amd.com%7Cc047ecb316df47cde7ed08d8de3188d4%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637503651624306464%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3f5ib%2FlZ6DXXF%2Bk1rXPKGu1IkOhXHdkUmX3obtuIRtA%3D&reserved=0">https://git-scm.com/docs/git-format-patch</a>]<o:p></o:p></pre> <pre> <o:p></o:p></pre> <pre>url: <a href="https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2F0day-ci%2Flinux%2Fcommits%2FOak-Zeng%2Fdrm-ttm-ioremap-buffer-according-to-TTM-mem-caching-setting%2F20210302-064500&data=04%7C01%7COak.Zeng%40amd.com%7Cc047ecb316df47cde7ed08d8de3188d4%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637503651624306464%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Y9h9ZHM7d2Sqo3XQWr%2Frpb2DAk9SkHFIm5nD4ABFNbE%3D&reserved=0">https://github.com/0day-ci/linux/commits/Oak-Zeng/drm-ttm-ioremap-buffer-according-to-TTM-mem-caching-setting/20210302-064500</a><o:p></o:p></pre> <pre>base: git://anongit.freedesktop.org/drm-intel for-linux-next<o:p></o:p></pre> <pre>config: parisc-randconfig-r012-20210302 (attached as .config)<o:p></o:p></pre> <pre>compiler: hppa-linux-gcc (GCC) 9.3.0<o:p></o:p></pre> <pre>reproduce (this is a W=1 build):<o:p></o:p></pre> <pre> wget <a href="https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fraw.githubusercontent.com%2Fintel%2Flkp-tests%2Fmaster%2Fsbin%2Fmake.cross&data=04%7C01%7COak.Zeng%40amd.com%7Cc047ecb316df47cde7ed08d8de3188d4%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637503651624316464%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=wGo3Nwk4mWZjb2W3E7qKf3FtKzLOxCVmQu1zd2rmzRU%3D&reserved=0">https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross</a> -O ~/bin/make.cross<o:p></o:p></pre> <pre> chmod +x ~/bin/make.cross<o:p></o:p></pre> <pre> # <a href="https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2F0day-ci%2Flinux%2Fcommit%2F225bb3711439ec559dd72ae5af8e62d34ea60a64&data=04%7C01%7COak.Zeng%40amd.com%7Cc047ecb316df47cde7ed08d8de3188d4%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637503651624316464%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=kOz18UAawmA4SZ6IrCGRt4HISfsmmjfGN2TUgAtm0Ak%3D&reserved=0">https://github.com/0day-ci/linux/commit/225bb3711439ec559dd72ae5af8e62d34ea60a64</a><o:p></o:p></pre> <pre> git remote add linux-review <a href="https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2F0day-ci%2Flinux&data=04%7C01%7COak.Zeng%40amd.com%7Cc047ecb316df47cde7ed08d8de3188d4%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637503651624326458%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=QnKs47gjzAz%2Fjv%2FAO6H%2BwYNQF6mdk1D8dCNveW0YHuM%3D&reserved=0">https://github.com/0day-ci/linux</a><o:p></o:p></pre> <pre> git fetch --no-tags linux-review Oak-Zeng/drm-ttm-ioremap-buffer-according-to-TTM-mem-caching-setting/20210302-064500<o:p></o:p></pre> <pre> git checkout 225bb3711439ec559dd72ae5af8e62d34ea60a64<o:p></o:p></pre> <pre> # save the attached .config to linux build tree<o:p></o:p></pre> <pre> COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=parisc <o:p></o:p></pre> <pre> <o:p></o:p></pre> <pre>If you fix the issue, kindly add following tag as appropriate<o:p></o:p></pre> <pre>Reported-by: kernel test robot <a href="mailto:lkp@intel.com"><lkp@intel.com></a><o:p></o:p></pre> <pre> <o:p></o:p></pre> <pre>All errors (new ones prefixed by >>):<o:p></o:p></pre> <pre> <o:p></o:p></pre> <pre> drivers/gpu/drm/ttm/ttm_bo_util.c: In function 'ttm_resource_ioremap':<o:p></o:p></pre> <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"> <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"> <pre>drivers/gpu/drm/ttm/ttm_bo_util.c:95:11: error: implicit declaration of function 'ioremap_cache'; did you mean 'ioremap_uc'? [-Werror=implicit-function-declaration]<o:p></o:p></pre> </blockquote> </blockquote> <pre> 95 | addr = ioremap_cache(mem->bus.offset, bus_size);<o:p></o:p></pre> <pre> | ^~~~~~~~~~~~~<o:p></o:p></pre> <pre> | ioremap_uc<o:p></o:p></pre> <pre> drivers/gpu/drm/ttm/ttm_bo_util.c:95:9: warning: assignment to 'void *' from 'int' makes pointer from integer without a cast [-Wint-conversion]<o:p></o:p></pre> <pre> 95 | addr = ioremap_cache(mem->bus.offset, bus_size);<o:p></o:p></pre> <pre> | ^<o:p></o:p></pre> <pre> drivers/gpu/drm/ttm/ttm_bo_util.c: In function 'ttm_bo_ioremap':<o:p></o:p></pre> <pre> drivers/gpu/drm/ttm/ttm_bo_util.c:379:17: warning: assignment to 'void *' from 'int' makes pointer from integer without a cast [-Wint-conversion]<o:p></o:p></pre> <pre> 379 | map->virtual = ioremap_cache(bo->mem.bus.offset + offset,<o:p></o:p></pre> <pre> | ^<o:p></o:p></pre> <pre> drivers/gpu/drm/ttm/ttm_bo_util.c: In function 'ttm_bo_vmap':<o:p></o:p></pre> <pre> drivers/gpu/drm/ttm/ttm_bo_util.c:500:16: warning: assignment to 'void *' from 'int' makes pointer from integer without a cast [-Wint-conversion]<o:p></o:p></pre> <pre> 500 | vaddr_iomem = ioremap_cache(mem->bus.offset,<o:p></o:p></pre> <pre> | ^<o:p></o:p></pre> <pre> cc1: some warnings being treated as errors<o:p></o:p></pre> <pre> <o:p></o:p></pre> <pre> <o:p></o:p></pre> <pre>vim +95 drivers/gpu/drm/ttm/ttm_bo_util.c<o:p></o:p></pre> <pre> <o:p></o:p></pre> <pre> 74 <o:p></o:p></pre> <pre> 75 static int ttm_resource_ioremap(struct ttm_bo_device *bdev,<o:p></o:p></pre> <pre> 76 struct ttm_resource *mem,<o:p></o:p></pre> <pre> 77 void **virtual)<o:p></o:p></pre> <pre> 78 {<o:p></o:p></pre> <pre> 79 int ret;<o:p></o:p></pre> <pre> 80 void *addr;<o:p></o:p></pre> <pre> 81 <o:p></o:p></pre> <pre> 82 *virtual = NULL;<o:p></o:p></pre> <pre> 83 ret = ttm_mem_io_reserve(bdev, mem);<o:p></o:p></pre> <pre> 84 if (ret || !mem->bus.is_iomem)<o:p></o:p></pre> <pre> 85 return ret;<o:p></o:p></pre> <pre> 86 <o:p></o:p></pre> <pre> 87 if (mem->bus.addr) {<o:p></o:p></pre> <pre> 88 addr = mem->bus.addr;<o:p></o:p></pre> <pre> 89 } else {<o:p></o:p></pre> <pre> 90 size_t bus_size = (size_t)mem->num_pages << PAGE_SHIFT;<o:p></o:p></pre> <pre> 91 <o:p></o:p></pre> <pre> 92 if (mem->bus.caching == ttm_write_combined)<o:p></o:p></pre> <pre> 93 addr = ioremap_wc(mem->bus.offset, bus_size);<o:p></o:p></pre> <pre> 94 else if (mem->bus.caching == ttm_cached)<o:p></o:p></pre> <pre> > 95 addr = ioremap_cache(mem->bus.offset, bus_size);<o:p></o:p></pre> <pre> 96 else<o:p></o:p></pre> <pre> 97 addr = ioremap(mem->bus.offset, bus_size);<o:p></o:p></pre> <pre> 98 if (!addr) {<o:p></o:p></pre> <pre> 99 ttm_mem_io_free(bdev, mem);<o:p></o:p></pre> <pre> 100 return -ENOMEM;<o:p></o:p></pre> <pre> 101 }<o:p></o:p></pre> <pre> 102 }<o:p></o:p></pre> <pre> 103 *virtual = addr;<o:p></o:p></pre> <pre> 104 return 0;<o:p></o:p></pre> <pre> 105 }<o:p></o:p></pre> <pre> 106 <o:p></o:p></pre> <pre> <o:p></o:p></pre> <pre>---<o:p></o:p></pre> <pre>0-DAY CI Kernel Test Service, Intel Corporation<o:p></o:p></pre> <pre><a href="https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.01.org%2Fhyperkitty%2Flist%2Fkbuild-all%40lists.01.org&data=04%7C01%7COak.Zeng%40amd.com%7Cc047ecb316df47cde7ed08d8de3188d4%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637503651624326458%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=MY6tEGVY0X4VyS750dYmsfVoR71Zp78YdxuqqRljnsI%3D&reserved=0">https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org</a><o:p></o:p></pre> <p class="MsoNormal"><br> <br> <br> <o:p></o:p></p> <pre>_______________________________________________<o:p></o:p></pre> <pre>amd-gfx mailing list<o:p></o:p></pre> <pre><a href="mailto:amd-gfx@lists.freedesktop.org">amd-gfx@lists.freedesktop.org</a><o:p></o:p></pre> <pre><a href="https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&data=04%7C01%7COak.Zeng%40amd.com%7Cc047ecb316df47cde7ed08d8de3188d4%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637503651624336455%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Or3tWmS5M6mXOaFP4FHsEbg0rqoizNGpTl6JEkD6fac%3D&reserved=0">https://lists.freedesktop.org/mailman/listinfo/amd-gfx</a><o:p></o:p></pre> </blockquote> <p class="MsoNormal"> <o:p></o:p></p> </div> </div> </blockquote> <p class="MsoNormal"><o:p> </o:p></p> </div> </div> </div> </body> </html>