[Mesa-users] Mesa XWindows memory leak causing application instability

[Chuck Atkins]

Hi Rick,
If you file a bug report it will usually get looked at by the necessary
devs.  If you don't hear anything back on the bug after a few days then try
the dev list directly.  You can also check the IRC channel #dri-devel on
freenode.

- Chuck

On Tue, Jun 13, 2017 at 12:10 PM, Rick Irons <Rick.Irons at mathworks.com>
wrote:

> Hi all,
>
>
>
> Any thoughts on the issue identified below?
>
>
>
> Perhaps this issue is something for the mesa-dev mailing list.
>
>
>
> Thanks,
>
> Rick
>
>
>
> *From:* Rick Irons
> *Sent:* Friday, June 9, 2017 5:59 PM
> *To:* mesa-users at lists.freedesktop.org
> *Subject:* Mesa XWindows memory leak causing application instability
>
>
>
> Hi,
>
>
>
> We are encountering sporadic application instability with an XWindows
> based implementation of Mesa.  The source of the instability appears to be
> a memory leak.  The details of our investigation are included below.
>
>
>
> This issue appears to have been introduced in Mesa 8.0.0 and is present in
> the current 17.1.2 release.
>
>
>
> Our workaround is identified below.  We are interested to know if this is
> a valid fix or if there is an alternative preferred approach to addressing
> the problem.
>
>
>
> Thanks for any help that you can provide.
>
>
>
> Rick
>
>
>
>
>
>
>
> *Problem description:*
>
> It looks like Mesa is dependent on MIT-SHM extension in the sense that if
> the MIT-SHM loads, it needs to be left in the same state such that Mesa's
> call to XShmDetatch() works correctly, with no unintended side-effects.
>
>
>
> The order of extension "close" callbacks appear to be driven by a loop in
> XCloseDisplay() where "Display *" points to a linked-list identified in the
> "ext_procs" field.
>
>
>
> We find that in Mesa’s current state, MIT-SHM's close_display extension
> hook is called _before_ Mesa's extension hook has been called. Thus,
> MIT-SHM's shm_info has removed/deleted the _XExtDisplayInfo entry
> associated with the Display * being closed _before_ Mesa has called
> XShmDetach as part of its close callback.
>
>
>
> The effect is to create a new _XExtDisplayInfo pointer that's left
> pointing to a freed "codes" field. If the _next_ caller of XOpenDisplay()
> is unlucky enough to a) obtain a recycled address of a "Display *" matching
> the previous one and b) call XShmQueryVersion(), there is a chance that the
> "codes" value(s) have been overwritten resulting in XError of BadRequest,
> BadLength, or hanging (all have been observed).  This appears to be what
> https://github.com/mesa3d/mesa/commit/0a20051e6da99e91b7bf58
> 9ea457c77a8b618f26 alludes to. In fact, I added XShmQueryVersion() back
> in to xm_api.c, removed the XShmQueryVersion() I'd incorporated into
> glxgears-based test program, and reproduced this failure:
>
>
>
> X Error of failed request:  BadRequest (invalid request code or no such
> operation)
>
> Major opcode of failed request:  0 ()
>
> Serial number of failed request:  17
>
> Current serial number in output stream:  17
>
>
>
> which is exactly what we see when our glxgears-based test program calls
> XShmQueryVersion() after a few loops.
>
>
>
> *Reproduction:*
>
> See the modified "glxgears" demo. If possible, run on Debian 8.3 and keep
> hitting escape to close the current gears demo window and advance the
> loop.
>
>
>
> *Possible workaround:*
>
> If src/mesa/drivers/x11/fakeglx.c references MIT-SHM _before_ registering
> itself as an extension with the current display, this appears to allow
> Mesa's call to XShmDetach to occur with no unintended consequence.
> MIT-SHM's close callback is called _after_ Mesa has performed its clean up
> and we see no stranded entries in shm_info associated with the current
> display.
>
>
>
> _______________________________________________
> mesa-users mailing list
> mesa-users at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/mesa-users/attachments/20170616/c387fa64/attachment.html>