<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <meta name="Generator" content="Microsoft Word 15 (filtered medium)"> <style></style> </head> <body lang="EN-US" link="blue" vlink="purple"> <div class="WordSection1"> <p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Hi Szilárd,<o:p></o:p></span></p> <p class="MsoNormal"><a name="_MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></a></p> <p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Since you have some questions and I could not reproduce them here I just make some response that I think may related to this problem:<o:p></o:p></span></p> <p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2"><![if !supportLists]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman""> </span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">We only have 64K Share local memory for all of the work groups<o:p></o:p></span></p> <p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2"><![if !supportLists]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><span style="mso-list:Ignore">2.<span style="font:7.0pt "Times New Roman""> </span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">The drm_intel_gem_bo_context_exec() failed have a lot of reasons, could give us the detail about your test about the execution wide?<o:p></o:p></span></p> <p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2"><![if !supportLists]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><span style="mso-list:Ignore">3.<span style="font:7.0pt "Times New Roman""> </span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">As far as I know most enqueue in beignet default to be blocking(some related to GPU is not blocking) , you can see that api clFlush is actually an empty function.<o:p></o:p></span></p> <p class="MsoNormal" style="margin-left:.25in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Also I am trying to reproduce your bug here and I am setting up GROMACS. Is there anything I should know to run it with beignet?<o:p></o:p></span></p> <p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p> <p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Thanks<o:p></o:p></span></p> <p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Xiuli <o:p></o:p></span></p> <p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p> <p class="MsoNormal"><a name="_____replyseparator"></a><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Beignet [mailto:beignet-bounces@lists.freedesktop.org] <b>On Behalf Of </b>Szilárd Páll<br> <b>Sent:</b> Thursday, March 31, 2016 3:14 AM<br> <b>To:</b> beignet@lists.freedesktop.org<br> <b>Subject:</b> Re: [Beignet] GROMACS on beignet<o:p></o:p></span></p> <p class="MsoNormal"><o:p> </o:p></p> <div> <p class="MsoNormal">Hello again,<o:p></o:p></p> <div> <p class="MsoNormal"><o:p> </o:p></p> </div> <div> <p class="MsoNormal">I have been trying to verify whether there may be assumptions >=32-wide execution hiding in the kernels (in particular in code that's using local memory for prefetching or reduction) and tried dropping in mem fences to test a few things, but at several points I managed to trigger the aforementioned error:<o:p></o:p></p> </div> <div> <div> <p class="MsoNormal">drm_intel_gem_bo_context_exec() failed: Input/output error<o:p></o:p></p> </div> </div> <div> <p class="MsoNormal"><o:p> </o:p></p> </div> <div> <p class="MsoNormal">Is this a known issues? There have been reports of it, but perhaps it is just the manifestation of multiple possible issues?<o:p></o:p></p> </div> <div> <p class="MsoNormal"><o:p> </o:p></p> </div> <div> <p class="MsoNormal">Secondly, I do not see the reason why I get blocking behavior of all enqueue operations (and I don't get this on NVIDIA or AMD). Are there any peculiarities I should be aware of?<o:p></o:p></p> </div> <div> <p class="MsoNormal"><o:p> </o:p></p> </div> <div> <p class="MsoNormal">Cheers,<o:p></o:p></p> </div> </div> <div> <p class="MsoNormal"><br clear="all"> <o:p></o:p></p> <div> <div> <p class="MsoNormal">--<br> Szilárd<o:p></o:p></p> </div> </div> <p class="MsoNormal"><o:p> </o:p></p> <div> <p class="MsoNormal">On Mon, Mar 28, 2016 at 1:49 AM, Szilárd Páll <<a href="mailto:sin.pecado@gmail.com" target="_blank">sin.pecado@gmail.com</a>> wrote:<o:p></o:p></p> <blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt"> <div> <p class="MsoNormal">Hi Xiuli,<o:p></o:p></p> <div> <p class="MsoNormal"><o:p> </o:p></p> </div> <div> <p class="MsoNormal">Thanks for the quick reply!<o:p></o:p></p> <div> <p class="MsoNormal"><o:p> </o:p></p> </div> <div> <div> <p class="MsoNormal">On Fri, Mar 25, 2016 at 4:06 AM, Pan, Xiuli <<a href="mailto:xiuli.pan@intel.com" target="_blank">xiuli.pan@intel.com</a>> wrote:<o:p></o:p></p> <blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt"> <div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Hi Szilárd,</span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">What do you mean about quoted includes?</span><o:p></o:p></p> </div> </div> </blockquote> <div> <p class="MsoNormal"><o:p> </o:p></p> </div> <div> <p class="MsoNormal">I mean -I"/path/to/headers" does not work, but -I/path/to/headers does.<o:p></o:p></p> </div> <div> <p class="MsoNormal"> <o:p></o:p></p> </div> <blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt"> <div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">If you mean the include in kernels, I think we may have some problem with that. The *.cl we used for clang actually was a copied tmp version stored not in where is used to be. So I think if you just put what need to be included in the old place, clang could not find it. You could try a workaround to pass “-I where/your/header/is” as a build option to clBuildProgram.</span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Then if you have some double types used on Haswell it may have some problem. The hardware for HSW does not support double very well as we have refined our double support to hardware then, so HSW may have some issues with double type. If it is not the problem with double float, you can send your kernel as an attachment or report a bug on our Bugzilla(</span><a href="https://bugs.freedesktop.org" target="_blank"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">https://bugs.freedesktop.org</span></a><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">) and we will tried to fix it.</span><o:p></o:p></p> </div> </div> </blockquote> <div> <p class="MsoNormal"><o:p> </o:p></p> </div> <div> <p class="MsoNormal">No double precision in the kernels. <o:p></o:p></p> </div> <div> <p class="MsoNormal"><o:p> </o:p></p> </div> <div> <p class="MsoNormal">For now I'll post here, I feel like a bug report may be an overkill - especially as I can't provide a full repro case that does not involve building the entire application.<o:p></o:p></p> </div> <div> <p class="MsoNormal"><o:p> </o:p></p> </div> <div> <p class="MsoNormal">I've attached a minimum set of source files that's needed to compile. We have pretty heavy preprocessor use that generates kernels for the different inputs / outputs / computation combinations, so one particular flavor that's known to produce incorrect results is generated compiling <a href="http://nbnxn_ocl_kernels.cl" target="_blank"> nbnxn_ocl_kernels.cl</a> with the following flags:<o:p></o:p></p> </div> <div> <p class="MsoNormal"><o:p> </o:p></p> </div> <div> <p class="MsoNormal">-D_WARPLESS_SOURCE_ -DGMX_OCL_FASTGEN -DEL_RF -DEELNAME=_ElecRF -DLJ_COMB_GEOM -DVDWNAME=_VdwLJCombGeom -DCENTRAL=22 -DNBNXN_GPU_NCLUSTER_PER_SUPERCLUSTER=8 -DNBNXN_GPU_CLUSTER_SIZE=8 -DNBNXN_GPU_JGROUP_SIZE=4 -DNBNXN_AVOID_SING_R2_INC=1.0e-12f<o:p></o:p></p> </div> <div> <p class="MsoNormal"><o:p> </o:p></p> </div> <div> <div> <p class="MsoNormal"><o:p> </o:p></p> </div> <div> <p class="MsoNormal">Additionally I had a closer look and so far I have observed three issues (additional to the minor include issue mentione before):<o:p></o:p></p> </div> <div> <p class="MsoNormal"><o:p> </o:p></p> </div> <div> <p class="MsoNormal">1. If I do a manual prefetch into local memory followed by a mem fence (seenbnxn_ocl_kernel_nowarp.clh line 339), I get the following error:<o:p></o:p></p> </div> <div> <p class="MsoNormal">drm_intel_gem_bo_context_exec() failed: Input/output error<o:p></o:p></p> </div> <div> <p class="MsoNormal">The next kernel call then fails with CL_OUT_OF_RESOURCES.<o:p></o:p></p> </div> <div> <p class="MsoNormal">Without the manual prefetch it works better, but...<o:p></o:p></p> </div> <div> <p class="MsoNormal"><o:p> </o:p></p> </div> <div> <p class="MsoNormal">2. The results produced by the kernel are still somewhat off. It could be that I missed a subtle detail and the kernels still do not conform to the hardware's execution model. I'm very familar with Intel's hardware and these kernels were originally designed for 32/64 wide execution.<o:p></o:p></p> </div> </div> <div> <p class="MsoNormal"><o:p> </o:p></p> </div> <div> <p class="MsoNormal">3. All task enqueue calls seem to be blocking. <o:p></o:p></p> </div> <div> <p class="MsoNormal"><o:p> </o:p></p> </div> <div> <p class="MsoNormal"><o:p> </o:p></p> </div> <div> <p class="MsoNormal">Thanks & Cheers,<o:p></o:p></p> </div> <div> <p class="MsoNormal">--<o:p></o:p></p> </div> <div> <p class="MsoNormal">Szilárd<o:p></o:p></p> </div> <div> <p class="MsoNormal"><o:p> </o:p></p> </div> <blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt"> <div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Thanks</span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Xiuli</span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a name="m_8161337532417221177_m_-165604280253327"></a><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Beignet [mailto:</span><a href="mailto:beignet-bounces@lists.freedesktop.org" target="_blank"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">beignet-bounces@lists.freedesktop.org</span></a><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">] <b>On Behalf Of </b>Szilárd Páll<br> <b>Sent:</b> Friday, March 25, 2016 7:16 AM<br> <b>To:</b> </span><a href="mailto:beignet@lists.freedesktop.org" target="_blank"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">beignet@lists.freedesktop.org</span></a><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><br> <b>Subject:</b> [Beignet] GROMACS on beignet</span><o:p></o:p></p> <div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi,<o:p></o:p></p> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> </div> <div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I am a developer of the GROMACS (<a href="http://www.gromacs.org" target="_blank">www.gromacs.org</a>) molecular dynamics simulation package. We have OpenCL offload for some of the compute-intensive kernels which that works very well on AMD. I wanted to assess how feasible is to use an Intel iGPU in GROMACS and after jumping through some hoops I got a 4.2 kernel and beignet master installed.<o:p></o:p></p> </div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> </div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Then I ran into the first minor issue: it seems that beignet does not accept quoted includes although AFAIK the double-quoted include paths should be accepted, but that did not work. No big deal, it doesn't work with Apple's OpenCL either, but I thought I'd ask.<o:p></o:p></p> </div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> </div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">However, the bigger issue is that running on Haswell (HD 4600, I think) the kernel produces results that are very off (while the very same source gives correct results on other platforms). I've not much time to dig deeper, but I thought I'd drop a mail maybe somebody is interested in helping out with tips or even tracking down where the issue is.<o:p></o:p></p> </div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> </div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Suggestions would be welcome!<o:p></o:p></p> </div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> </div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Cheers,<br clear="all"> <o:p></o:p></p> <div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">--<br> Szilárd<o:p></o:p></p> </div> </div> </div> </div> </div> </div> </div> </div> </div> </blockquote> </div> <p class="MsoNormal"><o:p> </o:p></p> </div> </div> </div> </blockquote> </div> <p class="MsoNormal"><o:p> </o:p></p> </div> </div> </body> </html>