<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> <style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style> </head> <body dir="ltr"> <div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> > <span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif; font-size: 14.6667px; background-color: rgb(255, 255, 255); display: inline !important">What are the exact use-cases for Mesa specific allocation data?</span><br style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif; font-size: 14.6667px; background-color: rgb(255, 255, 255)"> </div> <div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> <span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif; font-size: 14.6667px; background-color: rgb(255, 255, 255); display: inline !important"><br> </span></div> <div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> <span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif; font-size: 14.6667px; background-color: rgb(255, 255, 255); display: inline !important">In general, the use case is to implement Vulkan allocation callbacks. </span></div> <div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> <span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif; font-size: 14.6667px; background-color: rgb(255, 255, 255); display: inline !important"><br> </span></div> <div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> <span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif; font-size: 14.6667px; background-color: rgb(255, 255, 255); display: inline !important">Our particular use case is memory accounting: there is a fixed memory budget, so we need to keep track (not as an external tool but in the application itself) of all the memory consumed by every component. So components can't just call malloc, but instead other memory allocating functions, which ensure they stay within their budget.</span></div> <div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> <span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif; font-size: 14.6667px;"> </span></div> <div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> At any rate, I believe we'll pursue's Brian' u_trackmem.h approach for now.</div> <div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> <br> </div> <div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> <br> </div> <div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> <span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif; font-size: 14.6667px; background-color: rgb(255, 255, 255); display: inline !important">Jose</span></div> <div id="appendonsend"></div> <div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)"> <br> </div> <hr tabindex="-1" style="display:inline-block; width:98%"> <div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> mesa-dev <mesa-dev-bounces@lists.freedesktop.org> on behalf of Tamminen, Eero T <eero.t.tamminen@intel.com><br> <b>Sent:</b> Tuesday, May 12, 2020 19:08<br> <b>To:</b> mesa-dev@lists.freedesktop.org <mesa-dev@lists.freedesktop.org><br> <b>Subject:</b> Re: [Mesa-dev] RFC: Memory allocation on Mesa</font> <div> </div> </div> <div class="BodyFragment"><font size="2"><span style="font-size:11pt"> <div class="PlainText">Hi,<br> <br> On Tue, 2020-05-12 at 14:36 +0000, Jose Fonseca wrote:<br> > From: mesa-dev <mesa-dev-bounces@lists.freedesktop.org> on behalf of<br> > Tamminen, Eero T <eero.t.tamminen@intel.com><br> > I've done a lot of resource usage analysis at former job[1], but I've<br> > never had needed anything like that. malloc etc all reside in a<br> > separate shared library from Mesa, so calls to them always cross<br> > dynamic library boundary and therefore all of them can be caught with<br> > the dynamic linker features (LD_PRELOAD, LD_AUDIT...).<br> > <br> > True, one can easily intercept all mallocs using that sort of dynamic<br> > linking tricks, when doing full application interception. (I even<br> > have done some of the sort on <a href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjrfonseca%2Fmemtrail&data=02%7C01%7Cjfonseca%40vmware.com%7C7803192c4bf349748ccd08d7f69f8305%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637249037275021774&sdata=Ep9otfr5LTrE8w2a3S1E12MhFHkt8Xvawp%2BxLbrciQs%3D&reserved=0"> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjrfonseca%2Fmemtrail&data=02%7C01%7Cjfonseca%40vmware.com%7C7803192c4bf349748ccd08d7f69f8305%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637249037275021774&sdata=Ep9otfr5LTrE8w2a3S1E12MhFHkt8Xvawp%2BxLbrciQs%3D&reserved=0</a> ,<br> > mostly to hunt down memory leaks on LLVM.) But the goal here is to<br> > intercept the OpenGL/Vulkan driver malloc calls alone. Not the<br> > application mallocs. Which is difficult to segregate when doing<br> > whole application interception.<br> <br> Only reason to do this that I can think of would be to report some Mesa<br> specific metrics to an application at run-time. But that will need<br> careful thought on how *applications* are supposed to *use* that data,<br> otherwise it's just "garbage in, garbage out".<br> <br> What are the exact use-cases for Mesa specific allocation data?<br> <br> <br> > For simplicity imagine you have only these shared objects:<br> > application (not controlled by us)<br> > libVulkan.so<br> > libstdcc++.so<br> > libc.so<br> > <br> > Now imagine you're intercepting malloc doing some sort of LD_PRELOAD<br> > interception, and malloc is called. How do you know if it's a call<br> > done by the Vulkan driver, hence should call the callback, or one<br> > done by the application, hence not call the Vulkan allocation<br> > callback.<br> > <br> > One can look at the caller IP address, but what if the caller is in<br> > libstdc++ which is used both by Vulkan and the app, is not<br> > immediately clear which to bill the memory. One would need to walk<br> > back the stack completely, which is complicated and not very<br> > reliable.<br> <br> Over decade ago it was unreliable, but not anymore. Even stripped<br> binaries contain the frame information section (I think it's needed<br> e.g. for handling C++ exceptions). So nowadays it's simple to use, and<br> Glibc has a function for it.<br> <br> For analysis, you don't do filtering while tracing, as that's too<br> limiting, you do it in post-processing phase.<br> <br> <br> If you really need to do run-time per-library filtering, you could do<br> it based on the backtrace addresses, and whether they fall within the<br> address range where the library you're interested about is mapped.<br> <br> For that, you need to overload library loading, so you can catch when<br> Mesa gets loaded, and find out its (address-space-randomized) load<br> address range.<br> <br> <br> > Imagine one guesses wrong -- the malloc interceptor believes the<br> > malloc call is done by the Vulkan driver, and calls the application<br> > callback, which then calls malloc again, and the interceptor guesses<br> > wrong again, therefore an infinite recursion loop.<br> <br> If you find your own code addresses from the backtrace, STOP. :-)<br> <br> <br> > Could you be confusing this with trying to catch some Mesa specific<br> > function, where dynamic linker can catch only calls from application<br> > to<br> > Mesa, but not calls within Mesa library itself (as they don't cross<br> > dynamic library boundary)?<br> > <br> > My goal from the beginning is intercepting all mallocs/frees done by<br> > Mesa OpenGL/Vulkan driver, and only those.<br> <br> If this is for analysis, I would do it in post-processing phase (with<br> sp-rtrace). Just ask post-processor to filter-in allocation backtraces<br> going through the library I'm interested about.<br> <br> <br> ...<br> > Yes, indeed Linux has much better tooling for this than<br> > Windows. Note that memory debugging on Windows is just one of our<br> > needs. The other being able to run Mesa driver on an embedded system<br> > with fixed amount of memory (a separate budget for Mesa mallocs.)<br> <br> If that embedded target is still running something Linux-like, but<br> doesn't have enough memory for collecting the data, I would just stream<br> the data out of the device, or store it to a file. And do analysis in<br> post-processing phase.<br> <br> If that's not an option, I would do memory analysis on a system which<br> has reasonable tools, and "emulate" relevant embedded device functional<br> differences on it. If the issue isn't reproducible that way, it may<br> be thread / data race, which needs different tools (valgrind, mutrace<br> etc).<br> <br> <br> - Eero<br> <br> PS. In the maemo-tools project, there's also tool called "functracer"<br> which can on ARM & x86 attach to an already running process and start<br> collecting resource information. sp-rtrace tooling can post-process its<br> output.<br> <br> (It basically does in user-space with ptrace syscall, what ftrace does<br> in kernel space, which requires some arch-specific assembly, so<br> unfortunately it's not very portable.)<br> <br> _______________________________________________<br> mesa-dev mailing list<br> mesa-dev@lists.freedesktop.org<br> <a href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fmesa-dev&data=02%7C01%7Cjfonseca%40vmware.com%7C7803192c4bf349748ccd08d7f69f8305%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637249037275021774&sdata=NbuuZXYT%2FpPeI%2Fou%2F14KJCqSf9jAACNcOSM7YIehc%2BU%3D&reserved=0">https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fmesa-dev&data=02%7C01%7Cjfonseca%40vmware.com%7C7803192c4bf349748ccd08d7f69f8305%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637249037275021774&sdata=NbuuZXYT%2FpPeI%2Fou%2F14KJCqSf9jAACNcOSM7YIehc%2BU%3D&reserved=0</a><br> </div> </span></font></div> </body> </html>