[Mesa-users] Mesa XWindows memory leak causing application instability

[Brian Paul]

Rick,

Which Mesa driver are you using?  What's the output of glxinfo?

Both the old Mesa Xlib driver and the newer gallium softpipe/llvmpipe 
drivers use XShm and I'm not sure which one you're using.

-Brian

On 06/16/2017 01:28 PM, Chuck Atkins wrote:
> Hi Rick,
> If you file a bug report it will usually get looked at by the necessary
> devs.  If you don't hear anything back on the bug after a few days then
> try the dev list directly.  You can also check the IRC channel
> #dri-devel on freenode.
>
> - Chuck
>
> On Tue, Jun 13, 2017 at 12:10 PM, Rick Irons <Rick.Irons at mathworks.com
> <mailto:Rick.Irons at mathworks.com>> wrote:
>
>     Hi all,____
>
>     __ __
>
>     Any thoughts on the issue identified below? ____
>
>     __ __
>
>     Perhaps this issue is something for the mesa-dev mailing list.____
>
>     __ __
>
>     Thanks,____
>
>     Rick____
>
>     __ __
>
>     *From:* Rick Irons
>     *Sent:* Friday, June 9, 2017 5:59 PM
>     *To:* mesa-users at lists.freedesktop.org
>     <mailto:mesa-users at lists.freedesktop.org>
>     *Subject:* Mesa XWindows memory leak causing application instability____
>
>     __ __
>
>     Hi,____
>
>     __ __
>
>     We are encountering sporadic application instability with an
>     XWindows based implementation of Mesa.  The source of the
>     instability appears to be a memory leak.  The details of our
>     investigation are included below.____
>
>     __ __
>
>     This issue appears to have been introduced in Mesa 8.0.0 and is
>     present in the current 17.1.2 release.____
>
>     __ __
>
>     Our workaround is identified below.  We are interested to know if
>     this is a valid fix or if there is an alternative preferred approach
>     to addressing the problem.____
>
>     __ __
>
>     Thanks for any help that you can provide.____
>
>     __ __
>
>     Rick____
>
>     __ __
>
>     __ __
>
>     __ __
>
>     *Problem description:____*
>
>     It looks like Mesa is dependent on MIT-SHM extension in the sense
>     that if the MIT-SHM loads, it needs to be left in the same state
>     such that Mesa's call to XShmDetatch() works correctly, with no
>     unintended side-effects.____
>
>     __ __
>
>     The order of extension "close" callbacks appear to be driven by a
>     loop in XCloseDisplay() where "Display *" points to a linked-list
>     identified in the "ext_procs" field.____
>
>     __ __
>
>     We find that in Mesa’s current state, MIT-SHM's close_display
>     extension hook is called _before_ Mesa's extension hook has been
>     called. Thus, MIT-SHM's shm_info has removed/deleted the
>     _XExtDisplayInfo entry associated with the Display * being closed
>     _before_ Mesa has called XShmDetach as part of its close callback.____
>
>     __ __
>
>     The effect is to create a new _XExtDisplayInfo pointer that's left
>     pointing to a freed "codes" field. If the _next_ caller of
>     XOpenDisplay() is unlucky enough to a) obtain a recycled address of
>     a "Display *" matching the previous one and b) call
>     XShmQueryVersion(), there is a chance that the "codes" value(s) have
>     been overwritten resulting in XError of BadRequest, BadLength, or
>     hanging (all have been observed).  This appears to be what
>     https://github.com/mesa3d/mesa/commit/0a20051e6da99e91b7bf589ea457c77a8b618f26
>     <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_mesa3d_mesa_commit_0a20051e6da99e91b7bf589ea457c77a8b618f26&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=Ie7_encNUsqxbSRbqbNgofw0ITcfE8JKfaUjIQhncGA&m=sDFaHTXy_QwKdQ0mlORDT6cCzkRnbjEwsqTMtCQJVsY&s=WErP_2_7ibjVwY_O0qnVjtrreVBAWiiSUt7LW_qKVxg&e=>
>     alludes to. In fact, I added XShmQueryVersion() back in to xm_api.c,
>     removed the XShmQueryVersion() I'd incorporated into glxgears-based
>     test program, and reproduced this failure:____
>
>     __ __
>
>     X Error of failed request:  BadRequest (invalid request code or no
>     such operation)____
>
>     Major opcode of failed request:  0 ()____
>
>     Serial number of failed request:  17____
>
>     Current serial number in output stream:  17 ____
>
>     __ __
>
>     which is exactly what we see when our glxgears-based test program
>     calls XShmQueryVersion() after a few loops.____
>
>     __ __
>
>     *Reproduction:____*
>
>     See the modified "glxgears" demo. If possible, run on Debian 8.3 and
>     keep hitting escape to close the current gears demo window and
>     advance the loop. ____
>
>     __ __
>
>     *Possible workaround:____*
>
>     If src/mesa/drivers/x11/fakeglx.c references MIT-SHM _before_
>     registering itself as an extension with the current display, this
>     appears to allow Mesa's call to XShmDetach to occur with no
>     unintended consequence.  MIT-SHM's close callback is called _after_
>     Mesa has performed its clean up and we see no stranded entries in
>     shm_info associated with the current display.____
>
>     __ __
>
>
>     _______________________________________________
>     mesa-users mailing list
>     mesa-users at lists.freedesktop.org
>     <mailto:mesa-users at lists.freedesktop.org>
>     https://lists.freedesktop.org/mailman/listinfo/mesa-users
>     <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.freedesktop.org_mailman_listinfo_mesa-2Dusers&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=Ie7_encNUsqxbSRbqbNgofw0ITcfE8JKfaUjIQhncGA&m=sDFaHTXy_QwKdQ0mlORDT6cCzkRnbjEwsqTMtCQJVsY&s=T3N_tCx9oTUNZVXOu85-SlFOD-ULhZGi8Z-_zmCcPb8&e=>
>
>
>
>
> _______________________________________________
> mesa-users mailing list
> mesa-users at lists.freedesktop.org
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.freedesktop.org_mailman_listinfo_mesa-2Dusers&d=DwIGaQ&c=uilaK90D4TOVoH58JNXRgQ&r=Ie7_encNUsqxbSRbqbNgofw0ITcfE8JKfaUjIQhncGA&m=sDFaHTXy_QwKdQ0mlORDT6cCzkRnbjEwsqTMtCQJVsY&s=T3N_tCx9oTUNZVXOu85-SlFOD-ULhZGi8Z-_zmCcPb8&e=
>