[Mesa-dev] [PATCH] llvmpipe: Avoid deadlock when unloading opengl32.dll

Tom Stellard tom at stellard.net
Thu Nov 13 06:36:21 PST 2014 

On Thu, Nov 13, 2014 at 11:10:39AM +0000, Jose Fonseca wrote:
> Hi Tom,
> 
> That's peculiar. It looks like pthreads got into a weird state somehow.  Don't precisely understand how though.  Maybe there's a race inside pipe_semaphore_signal() with the destruction of the semaphore.
> 
> I think the best thing for now is to revert to old behavior for non-windows platforms:
> 
> diff --git a/src/gallium/drivers/llvmpipe/lp_rast.c b/src/gallium/drivers/llvmpipe/lp_rast.c
> index 6b54d43..e168766 100644
> --- a/src/gallium/drivers/llvmpipe/lp_rast.c
> +++ b/src/gallium/drivers/llvmpipe/lp_rast.c
> @@ -800,7 +800,9 @@ static PIPE_THREAD_ROUTINE( thread_function, init_data )
>        pipe_semaphore_signal(&task->work_done);
>     }
>  
> +#ifdef _WIN32
>     pipe_semaphore_signal(&task->work_done);
> +#endif
>  
>     return 0;
>  }
> @@ -891,7 +893,11 @@ void lp_rast_destroy( struct lp_rasterizer *rast )
>      * We don't actually call pipe_thread_wait to avoid dead lock on Windows
>      * per https://bugs.freedesktop.org/show_bug.cgi?id=76252 */
>     for (i = 0; i < rast->num_threads; i++) {
> +#ifdef _WIN32
>        pipe_semaphore_wait(&rast->tasks[i].work_done);
> +#else
> +      pipe_thread_wait(rast->threads[i]);
> +#endif
>     }
>  
>     /* Clean up per-thread data */
> 
> 
> Because I don't think that the Windows deadlock ever happens on Linux.
> 

This solution works for me.  Feel free to commit.

I wonder if the problem may be the pipe-loader is unloading pipe_swrast.so
before all the threads have finished.

-Tom


> Jose
> 
> 
> ________________________________________
> From: Tom Stellard <tom at stellard.net>
> Sent: 13 November 2014 01:45
> To: Jose Fonseca
> Cc: mesa-dev at lists.freedesktop.org; Roland Scheidegger
> Subject: Re: [Mesa-dev] [PATCH] llvmpipe: Avoid deadlock when unloading opengl32.dll
> 
> On Fri, Nov 07, 2014 at 04:52:25PM +0000, jfonseca at vmware.com wrote:
> > From: José Fonseca <jfonseca at vmware.com>
> >
> 
> Hi Jose,
> 
> This patch is causing random segfaults with OpenCL programs on radeonsi.
> I haven't been able to figure out exactly what is happening, so I was
> hoping you could help.
> 
> I think the problem has something to do with the fact that when clover
> probes the hardware for OpenCL devices, the pipe_loader creates an
> llvmpipe screen, checks the value of PIPE_CAP_COMPUTE, and then destroys
> the screen since PIPE_CAP_COMPUTE is 0.
> 
> The only way I can reproduce this bug is by running the piglit OpenCL
> tests concurrently.  If it helps, here are the stack traces
> from one of the core dumps I captured from a piglit run:
> 
> (gdb) thread 1
> [Switching to thread 1 (Thread 0x7f6d53cdf700 (LWP 18653))]
> #0  0x00007f6d53e56d2d in ?? ()
> (gdb) bt
> #0  0x00007f6d53e56d2d in ?? ()
> #1  0x0000000000000000 in ?? ()
> (gdb) thread 2
> [Switching to thread 2 (Thread 0x7f6d5495f700 (LWP 18652))]
> #0  0x00007f6d5aacd44c in pthread_cond_wait () from /lib64/libpthread.so.0
> (gdb) bt
> #0  0x00007f6d5aacd44c in pthread_cond_wait () from /lib64/libpthread.so.0
> #1  0x00007f6d54c71dbb in mtx_init (mtx=0x7f6d54c71dbb <mtx_init+97>,type=0) at ../../../../../include/c11/threads_posix.h:182
> #2  0x00007f6d54c72157 in radeon_set_fd_access (applier=0x61e828,owner=0x61e800, mutex=0x7f6d54c71dbb <mtx_init+97>, request=0,request_name=0x0, enable=238 '\356') at radeon_drm_winsys.c:70
> #3  0x00007f6d54c7ad30 in radeon_drm_cs_emit_ioctl (param=0x61e4f0) at radeon_drm_winsys.c:598
> #4  0x00007f6d54c71ce0 in cnd_wait (cond=0x61e4f0, mtx=0x7f6d54c7ad07 <radeon_drm_cs_emit_ioctl+168>) at ../../../../../include/c11/threads_posix.h:152
> #5  0x00007f6d5aac91da in start_thread () from /lib64/libpthread.so.0
> #6  0x00007f6d5afd5d7d in clone () from /lib64/libc.so.6
> (gdb) thread 3
> [Switching to thread 3 (Thread 0x7f6d5c20c740 (LWP 18649))]
> #0  0x00007f6d5afae73e in re_node_set_insert_last () from /lib64/libc.so.6
> (gdb) bt
> #0  0x00007f6d5afae73e in re_node_set_insert_last () from /lib64/libc.so.6
> #1  0x00007f6d5afae7fe in register_state () from /lib64/libc.so.6
> #2  0x00007f6d5afb1d39 in re_acquire_state_context () from /lib64/libc.so.6
> #3  0x00007f6d5afbaa95 in re_compile_internal () from /lib64/libc.so.6
> #4  0x00007f6d5afbb603 in regcomp () from /lib64/libc.so.6
> #5  0x0000000000403e9b in regex_get_matches (src=0x63e6c0 "float", pattern=0x40b940 "^ulong|ulong2|ulong3|ulong4|ulong8|ulong16$", pmatch=0x0, size=0, cflags=4) at /home/tstellar/piglit/tests/cl/program/program-tester.c:476
> #6  0x00000000004040e2 in regex_match (src=0x63e6c0 "float", pattern=0x40b940 "^ulong|ulong2|ulong3|ulong4|ulong8|ulong16$") at /home/tstellar/piglit/tests/cl/program/program-tester.c:532
> #7  0x00000000004059c6 in get_test_arg (src=0x63de70 "1 buffer float[7] 0.5 -0.5 0.0 -0.0 nan -3.99 1.5", test=0x645710, arg_in=true) at /home/tstellar/piglit/tests/cl/program/program-tester.c:1016
> #8  0x0000000000406f4a in parse_config (     config_str=0x63fe30 "\n[config]\nname: Test float trunc built-in on CL 1.1\nclc_version_min: 10\ndimensions: 1\n\n[test]\nname: trunc float1\nkernel_name: test_1_trunc_float\nglobal_size: 7 0 0\n\narg_out: 0 buffer float[7] 0.0 -0.0"..., config=0x60e260 <config>) at /home/tstellar/piglit/tests/cl/program/program-tester.c:1410
> #9  0x00000000004074a7 in init (argc=2, argv=0x7fff46612d88, config=0x60e260 <config>) at /home/tstellar/piglit/tests/cl/program/program-tester.c:1555
> #10 0x00007f6d5be0232c in piglit_cl_program_test_init (argc=2, argv=0x7fff46612d88, void_config=0x60e260 <config>) at /home/tstellar/piglit/tests/util/piglit-framework-cl-program.c:60
> #11 0x00007f6d5be00f33 in piglit_cl_framework_run (argc=2, argv=0x7fff46612d88) at /home/tstellar/piglit/tests/util/piglit-framework-cl.c:154
> #12 0x0000000000403535 in main (argc=2, argv=0x7fff46612d88) at /home/tstellar/piglit/tests/cl/program/program-tester.c:164
> 
> 
> Thanks,
> Tom
> 
> > On Windows, DllMain calls and thread creation/destruction are
> > serialized, so when llvmpipe is destroyed from DllMain waiting for the
> > rasterizer threads to finish will deadlock.
> >
> > So, instead of waiting for rasterizer threads to have finished, simply wait for the
> > rasterizer threads to notify they are just about to finish.
> >
> > Verified with this very simple program:
> >
> >    #include <windows.h>
> >    int main() {
> >       HMODULE hModule = LoadLibraryA("opengl32.dll");
> >       FreeLibrary(hModule);
> >    }
> >
> > Fixes https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.freedesktop.org_show-5Fbug.cgi-3Fid-3D76252&d=AAIFAw&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=zfmBZnnVGHeYde45pMKNnVyzeaZbdIqVLprmZCM2zzE&m=4Id8ETfWCgJJvW3xybyB1vrW9ODBP9N_-G0is9bVbcU&s=qfRGj0dAovk-hsV3gL4r54_d-vlULQMia4rgMFLdOs0&e=
> >
> > Cc: 10.2 10.3 <mesa-stable at lists.freedesktop.org>
> > ---
> >  src/gallium/drivers/llvmpipe/lp_rast.c | 8 ++++++--
> >  1 file changed, 6 insertions(+), 2 deletions(-)
> >
> > diff --git a/src/gallium/drivers/llvmpipe/lp_rast.c b/src/gallium/drivers/llvmpipe/lp_rast.c
> > index a3420a2..6b54d43 100644
> > --- a/src/gallium/drivers/llvmpipe/lp_rast.c
> > +++ b/src/gallium/drivers/llvmpipe/lp_rast.c
> > @@ -800,6 +800,8 @@ static PIPE_THREAD_ROUTINE( thread_function, init_data )
> >        pipe_semaphore_signal(&task->work_done);
> >     }
> >
> > +   pipe_semaphore_signal(&task->work_done);
> > +
> >     return 0;
> >  }
> >
> > @@ -885,9 +887,11 @@ void lp_rast_destroy( struct lp_rasterizer *rast )
> >        pipe_semaphore_signal(&rast->tasks[i].work_ready);
> >     }
> >
> > -   /* Wait for threads to terminate before cleaning up per-thread data */
> > +   /* Wait for threads to terminate before cleaning up per-thread data.
> > +    * We don't actually call pipe_thread_wait to avoid dead lock on Windows
> > +    * per https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.freedesktop.org_show-5Fbug.cgi-3Fid-3D76252&d=AAIFAw&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=zfmBZnnVGHeYde45pMKNnVyzeaZbdIqVLprmZCM2zzE&m=4Id8ETfWCgJJvW3xybyB1vrW9ODBP9N_-G0is9bVbcU&s=qfRGj0dAovk-hsV3gL4r54_d-vlULQMia4rgMFLdOs0&e=  */
> >     for (i = 0; i < rast->num_threads; i++) {
> > -      pipe_thread_wait(rast->threads[i]);
> > +      pipe_semaphore_wait(&rast->tasks[i].work_done);
> >     }
> >
> >     /* Clean up per-thread data */
> > --
> > 1.9.1
> >
> > _______________________________________________
> > mesa-dev mailing list
> > mesa-dev at lists.freedesktop.org
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.freedesktop.org_mailman_listinfo_mesa-2Ddev&d=AAIFAw&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=zfmBZnnVGHeYde45pMKNnVyzeaZbdIqVLprmZCM2zzE&m=4Id8ETfWCgJJvW3xybyB1vrW9ODBP9N_-G0is9bVbcU&s=-ewFLSojDiybFSLUfQiK9J6LQJf4z7Mm_jpcDXschKI&e=

More information about the mesa-dev mailing list