<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> <style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style> </head> <body dir="ltr"> <div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> <br> </div> <div> <div id="appendonsend"></div> <div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)"> <br> </div> <hr tabindex="-1" style="display:inline-block; width:98%"> <div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> mesa-dev <mesa-dev-bounces@lists.freedesktop.org> on behalf of Tamminen, Eero T <eero.t.tamminen@intel.com><br> <b>Sent:</b> Monday, May 11, 2020 21:19<br> <b>To:</b> mesa-dev@lists.freedesktop.org <mesa-dev@lists.freedesktop.org><br> <b>Subject:</b> Re: [Mesa-dev] RFC: Memory allocation on Mesa</font> <div> </div> </div> <div class="BodyFragment"><font size="2"><span style="font-size:11pt"> <div class="PlainText"> <blockquote style="border-left: 3px solid rgb(200, 200, 200); border-top-color: rgb(200, 200, 200); border-right-color: rgb(200, 200, 200); border-bottom-color: rgb(200, 200, 200); padding-left: 1ex; margin-left: 0.8ex; color: rgb(102, 102, 102);"> Hi,<br> <br> On Mon, 2020-05-11 at 16:13 +0000, Jose Fonseca wrote:<br> > Some might retort: why not just play some tricks with the linker, and<br> > intercept all malloc/free calls, without actually having to modify<br> > any source code?<br> > <br> > Yes, that's indeed technically feasible. And is precisely the sort<br> > of trick I was planing to resort to satisfy VMware needs without<br> > having to further modify the source code. But for these tricks to<br> > work, it is absolutely imperative that one statically links C++<br> > library and STL. The problem being that if one doesn't then there's<br> > an imbalance: the malloc/free/new/delete calls done in inline code on<br> > C++ headers will be intercepted, where as malloc/free/new/delete<br> > calls done in code from the shared object which is not inlined will<br> > not, causing havoc. This is OK for us VMware (we do it already for<br> > many other reasons, including avoiding DLL hell.) But I doubt it<br> > will be palatable for everybody else, particularly Linux distros, to<br> > have everything statically linked.<br> <br> Huh?<br> <br> I've done a lot of resource usage analysis at former job[1], but I've<br> never had needed anything like that. malloc etc all reside in a<br> separate shared library from Mesa, so calls to them always cross<br> dynamic library boundary and therefore all of them can be caught with<br> the dynamic linker features (LD_PRELOAD, LD_AUDIT...).<br> <div><br> </div> </blockquote> <div>True, one can easily intercept all mallocs using that sort of dynamic linking tricks, when doing <i>full</i> application interception. (I even have done some of the sort on <a href="https://github.com/jrfonseca/memtrail"> https://github.com/jrfonseca/memtrail</a> , mostly to hunt down memory leaks on LLVM.) But the goal here is to intercept the OpenGL/Vulkan driver malloc calls alone. Not the application mallocs. Which is difficult to segregate when doing whole application interception.</div> <div><br> </div> <div>For simplicity imagine you have only these shared objects:</div> <div><br> </div> <div> application (not controlled by us)</div> <div> libVulkan.so</div> <div> libstdcc++.so</div> <div> libc.so</div> <div><br> </div> <div>Now imagine you're intercepting malloc doing some sort of LD_PRELOAD interception, and malloc is called. How do you know if it's a call done by the Vulkan driver, hence should call the callback, or one done by the application, hence not call the Vulkan allocation callback.</div> <div><br> </div> <div>One can look at the caller IP address, but what if the caller is in libstdc++ which is used both by Vulkan and the app, is not immediately clear which to bill the memory. One would need to walk back the stack completely, which is complicated and not very reliable.</div> <div><br> </div> <div>Imagine one guesses wrong -- the malloc interceptor believes the malloc call is done by the Vulkan driver, and calls the application callback, which then calls malloc again, and the interceptor guesses wrong again, therefore an infinite recursion loop.</div> <div><br> </div> <blockquote style="border-left: 3px solid rgb(200, 200, 200); border-top-color: rgb(200, 200, 200); border-right-color: rgb(200, 200, 200); border-bottom-color: rgb(200, 200, 200); padding-left: 1ex; margin-left: 0.8ex; color: rgb(102, 102, 102);"> <div><span style="color: rgb(102, 102, 102);">Could you be confusing this with trying to catch some Mesa specific</span><br> </div> function, where dynamic linker can catch only calls from application to<br> Mesa, but not calls within Mesa library itself (as they don't cross<br> <div>dynamic library boundary)?<br> </div> </blockquote> <div><br> </div> <div>My goal from the beginning is intercepting all mallocs/frees done by Mesa OpenGL/Vulkan driver, and only those. </div> <div><br> </div> <blockquote style="border-left: 3px solid rgb(200, 200, 200); border-top-color: rgb(200, 200, 200); border-right-color: rgb(200, 200, 200); border-bottom-color: rgb(200, 200, 200); padding-left: 1ex; margin-left: 0.8ex; color: rgb(102, 102, 102);"> Note: at least earlier, new & delete typically called malloc & free (in<br> addition to calling ctor & dtor), in which case you don't even need to<br> track them separately. You see their usage directly from the<br> allocation callgraph.<br> <br> <br> - Eero<br> <br> PS. Your XDot tool was a really nice tool for viewing those call-<br> <div>graphs. :-)<br> </div> <blockquote style="border-left: 3px solid rgb(200, 200, 200); border-top-color: rgb(200, 200, 200); border-right-color: rgb(200, 200, 200); border-bottom-color: rgb(200, 200, 200); padding-left: 1ex; margin-left: 0.8ex; color: rgb(102, 102, 102);"> <br> </blockquote> </blockquote> <div>Thanks! :)</div> <blockquote style="border-left: 3px solid rgb(200, 200, 200); border-top-color: rgb(200, 200, 200); border-right-color: rgb(200, 200, 200); border-bottom-color: rgb(200, 200, 200); padding-left: 1ex; margin-left: 0.8ex; color: rgb(102, 102, 102);"> <br> [1] Linux has several ready-made tools for tracking resource<br> allocations (several Valgrind tools, ElectricFence, Duma etc), and we<br> added few more at Nokia, with main one being:<br> <a href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmaemo-tools-old%2Fsp-rtrace&data=02%7C01%7Cjfonseca%40vmware.com%7Cc77b32234e7c4e59af7f08d7f5e89d09%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637248251736102860&sdata=4ZD66qaYmgCBv%2FQqRqamCWh1wYdFxqvuz0sqDZEnC3Y%3D&reserved=0">https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmaemo-tools-old%2Fsp-rtrace&data=02%7C01%7Cjfonseca%40vmware.com%7Cc77b32234e7c4e59af7f08d7f5e89d09%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637248251736102860&sdata=4ZD66qaYmgCBv%2FQqRqamCWh1wYdFxqvuz0sqDZEnC3Y%3D&reserved=0</a><br> <br> (Most memorable thing was early Qt/C++ application version doing<br> ~100000(!) allocation frees while it was initializing itself, due to<br> redundantly creating, localizing and removing one user view.)<br> </blockquote> <div class="PlainText">Yes, indeed Linux has much better tooling for this than Windows. Note that memory debugging on Windows is just one of our needs. The other being able to run Mesa driver on an embedded system with fixed amount of memory (a separate budget for Mesa mallocs.)</div> <div class="PlainText"><br> </div> <div class="PlainText">Jose</div> <br> ____________________________________________<br> </div> </span></font></div> </div> </body> </html>