gstreamer and valgrind
Krzysztof Borowczyk
k.borowczyk at samsung.com
Thu Aug 21 05:15:23 PDT 2014
Hello,
Just an update on the topic - I tested today on current, 1.4.0 branch and the problem doesn't appear, it seems it's been fixed since 1.2.4.
--
Best regards,
Krzysztof Borowczyk
> -----Original Message-----
> From: gstreamer-devel [mailto:gstreamer-devel-
> bounces at lists.freedesktop.org] On Behalf Of Krzysztof Borowczyk
> Sent: Wednesday, August 20, 2014 5:06 PM
> To: 'Discussion of the development of and with GStreamer'
> Subject: RE: gstreamer and valgrind
>
> Hello,
>
> I've composed a sample GStreamer pipeline that leaks, although it
> doesn't leak as much as in original problem, also it stops leaking
> before running out of memory.
>
> The command is:
>
> gst-launch-1.0 udpsrc uri="my_ip:5555" ! queue ! decodebin !
> videoconvert ! avenc_flv ! flvmux streamable=true ! rtmpsink
> location="rtmp://rtmp_server:8080/oflaDemo/streamName live=1"
>
> The memory as shown by htop (VIRT/RES/SHR):
> 1. "No signal" banner: 515M/25212/6308
> 2. Signal turned on: 517M/26532/6312
> 3. "No signal" again: 719M/228M/6356
> 4. Signal turned on: 720M/230M/6356
> 5. "No signal" again: 913M/424M/6456
>
>
> On one run I got SIGSEGV on step 3. The backtrace was:
> (gdb) back
> #0 0x00007f8c248ee03d in poll () at ../sysdeps/unix/syscall-
> template.S:81
> #1 0x00007f8c24e2cfe4 in ?? () from /lib/x86_64-linux-gnu/libglib-
> 2.0.so.0
> #2 0x00007f8c24e2d30a in g_main_loop_run ()
> from /lib/x86_64-linux-gnu/libglib-2.0.so.0
> #3 0x00007f8c25378a94 in gst_bus_poll (bus=bus at entry=0x2534b70,
> events=events at entry=GST_MESSAGE_ANY, timeout=<optimized out>)
> at gstbus.c:1082
> #4 0x00000000004045a8 in event_loop (pipeline=0x2722040,
> blocking=blocking at entry=1, do_progress=do_progress at entry=0,
> target_state=target_state at entry=GST_STATE_PLAYING) at gst-
> launch.c:509
> #5 0x0000000000403767 in main (argc=17, argv=0x7fff093ce3c8)
> at gst-launch.c:1085
>
>
> I saved the core in case some other details would be helpful (it's
> 185M).
>
> When run in Valgrind this pipe grows even without switching the signal.
> I waited until it got to about 1500M(RES), then did the signal
> switching. This is what Valgrind says:
>
> ==14224== 265,263,534 (63,240 direct, 265,200,294 indirect) bytes in
> 465 blocks are definitely lost in loss record 3,880 of 3,882
> ==14224== at 0x4C2AB80: malloc (in
> /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==14224== by 0x53DA610: g_malloc (in /lib/x86_64-linux-gnu/libglib-
> 2.0.so.0.4000.0)
> ==14224== by 0x53F022D: g_slice_alloc (in /lib/x86_64-linux-
> gnu/libglib-2.0.so.0.4000.0)
> ==14224== by 0x4E6E36A: gst_buffer_add_meta (gstbuffer.c:1915)
> ==14224== by 0x9121760: gst_buffer_add_video_meta_full
> (gstvideometa.c:231)
> ==14224== by 0x91227A8: video_buffer_pool_alloc (gstvideopool.c:202)
> ==14224== by 0x4E6F353: do_alloc_buffer.constprop.2
> (gstbufferpool.c:272)
> ==14224== by 0x4E6F6F5: default_acquire_buffer (gstbufferpool.c:995)
> ==14224== by 0x4E70A01: gst_buffer_pool_acquire_buffer
> (gstbufferpool.c:1099)
> ==14224== by 0x7A0D63A: default_prepare_output_buffer
> (gstbasetransform.c:1577)
> ==14224== by 0x7A0F3BA: gst_base_transform_handle_buffer
> (gstbasetransform.c:2065)
> ==14224== by 0x7A0FD00: gst_base_transform_chain
> (gstbasetransform.c:2201)
> ==14224==
> ==14224== 1,392,821,707 (546,744 direct, 1,392,274,963 indirect) bytes
> in 2,071 blocks are definitely lost in loss record 3,881 of 3,882
> ==14224== at 0x4C2AB80: malloc (in
> /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==14224== by 0x53DA610: g_malloc (in /lib/x86_64-linux-gnu/libglib-
> 2.0.so.0.4000.0)
> ==14224== by 0x53F022D: g_slice_alloc (in /lib/x86_64-linux-
> gnu/libglib-2.0.so.0.4000.0)
> ==14224== by 0x53F076D: g_slice_alloc0 (in /lib/x86_64-linux-
> gnu/libglib-2.0.so.0.4000.0)
> ==14224== by 0x912E699: gst_video_encoder_chain
> (gstvideoencoder.c:1212)
> ==14224== by 0x4E97F07: gst_pad_push_data (gstpad.c:3760)
> ==14224== by 0x7A0FF08: gst_base_transform_chain
> (gstbasetransform.c:2237)
> ==14224== by 0x4E97F07: gst_pad_push_data (gstpad.c:3760)
> ==14224== by 0x4E89F0A: gst_proxy_pad_chain_default
> (gstghostpad.c:128)
> ==14224== by 0x4E97F07: gst_pad_push_data (gstpad.c:3760)
> ==14224== by 0x91252BE: gst_video_decoder_clip_and_push_buf
> (gstvideodecoder.c:2657)
> ==14224== by 0x912A447: gst_video_decoder_finish_frame
> (gstvideodecoder.c:2572)
> ==14224==
> ==14224== 1,653,911,262 bytes in 2,658 blocks are indirectly lost in
> loss record 3,882 of 3,882
> ==14224== at 0x4C2AB80: malloc (in
> /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==14224== by 0x53DA610: g_malloc (in /lib/x86_64-linux-gnu/libglib-
> 2.0.so.0.4000.0)
> ==14224== by 0x53F022D: g_slice_alloc (in /lib/x86_64-linux-
> gnu/libglib-2.0.so.0.4000.0)
> ==14224== by 0x4E627CC: _sysmem_new_block (gstallocator.c:415)
> ==14224== by 0x4E6C471: gst_buffer_new_allocate (gstbuffer.c:668)
> ==14224== by 0x912275C: video_buffer_pool_alloc (gstvideopool.c:195)
> ==14224== by 0x4E6F353: do_alloc_buffer.constprop.2
> (gstbufferpool.c:272)
> ==14224== by 0x4E6F6F5: default_acquire_buffer (gstbufferpool.c:995)
> ==14224== by 0x4E70A01: gst_buffer_pool_acquire_buffer
> (gstbufferpool.c:1099)
> ==14224== by 0x7A0D63A: default_prepare_output_buffer
> (gstbasetransform.c:1577)
> ==14224== by 0x7A0F3BA: gst_base_transform_handle_buffer
> (gstbasetransform.c:2065)
> ==14224== by 0x7A0FD00: gst_base_transform_chain
> (gstbasetransform.c:2201)
> ==14224==
> ==14224== LEAK SUMMARY:
> ==14224== definitely lost: 626,733 bytes in 2,548 blocks
> ==14224== indirectly lost: 1,657,475,257 bytes in 9,597 blocks
> ==14224== possibly lost: 136,289,663 bytes in 600 blocks
> ==14224== still reachable: 995,020 bytes in 7,242 blocks
> ==14224== suppressed: 0 bytes in 0 blocks
> ==14224==
> ==14224== For counts of detected and suppressed errors, rerun with: -v
> ==14224== Use --track-origins=yes to see where uninitialised values
> come from
> ==14224== ERROR SUMMARY: 314 errors from 270 contexts (suppressed: 0
> from 0)
>
> --
> Best regards,
> Krzysztof Borowczyk
>
> > -----Original Message-----
> > From: gstreamer-devel [mailto:gstreamer-devel-
> > bounces at lists.freedesktop.org] On Behalf Of Krzysztof Borowczyk
> > Sent: Wednesday, August 20, 2014 3:52 PM
> > To: 'Discussion of the development of and with GStreamer'
> > Subject: RE: gstreamer and valgrind
> >
> > [...]
> >
> > > >
> > > > GStreamer now works differently
> > >
> > > Note that the message says that it *might* now take different code
> > > paths
> > > to ease debugging. As far as I know, it doesn't actually do that in
> > > practice, besides printing that info text.
> > >
> > > > and I get leaks quite rarely.
> > >
> > > Are you sure about that? Perhaps it just leaks slower because it
> runs
> > > slower?
> >
> > Yes it runs and leaks much slower, but it has been a bit more
> difficult
> > to trigger the leak. The way to trigger it is quite strange
> (explained
> > below).
> >
> > The setup is:
> >
> > 1. First computer: ffmpeg streaming mpegts to
> udp://secondComputer:5555
> > - ffmpeg grabs input of avermedia grabbing card.
> > 2. On the second one the (simplified) pipeline is:
> > udpsrc -> queue -> decodebin -> [...] -> avenc_flv -> flvmux ->
> > rtmpsink
> >
> > I've run similar pipelines with different sources (left of [...])
> > without noticing any serious leakage. (shmsink, decklinksrc,
> filesrc).
> >
> > To trigger the leak:
> > 1. Start the pipeline with the stream content being video of static
> > image (No signal panel from avermedia card on the other computer)
> > 2. Start video input on the avermedia, so it's no longer static
> image.
> > 3. After some time turn off the signal, video goes back to static
> > image.
> > 4. Huge leaking starts now.
> >
> > It's a way that triggers the leak 100% of the time. If I stop at
> point
> > 1 or 2 there seems to be no significant leak during 10min test period
> -
> > leaking starts immediately after switching the signal off.
> >
> > I suppose this switching of signal on the first computer should have
> no
> > influence on GStreamer, ffmpeg is streaming all the time, just
> > different content - do you have any idea why this might happen? I'm
> > using GStreamer 1.2.4, were there any significant leak fixes in the
> > meantime?
> >
> > I will try to prepare a test case with clean GStreamer, without our
> > application. In the meantime this is the bottom of Valgrind log
> without
> > G_SLICE=always-malloc, below that one is another one, with G_SLICE
> set.
> > Application was stopped with ctrl-c, unfortunately there is no easy
> way
> > to terminate it normally.
> >
> > ==8282== 174,974,310 bytes in 90 blocks are possibly lost in loss
> > record 9,438 of 9,440
> > ==8282== at 0x4C2AB80: malloc (in
> > /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> > ==8282== by 0x4E85610: g_malloc (in /lib/x86_64-linux-gnu/libglib-
> > 2.0.so.0.4000.0)
> > ==8282== by 0x4E9B22D: g_slice_alloc (in /lib/x86_64-linux-
> > gnu/libglib-2.0.so.0.4000.0)
> > ==8282== by 0x63B87CC: _sysmem_new_block (gstallocator.c:415)
> > ==8282== by 0x63C2471: gst_buffer_new_allocate (gstbuffer.c:668)
> > ==8282== by 0x289F275C: video_buffer_pool_alloc
> (gstvideopool.c:195)
> > ==8282== by 0x63C5353: do_alloc_buffer.constprop.2
> > (gstbufferpool.c:272)
> > ==8282== by 0x63C56F5: default_acquire_buffer
> (gstbufferpool.c:995)
> > ==8282== by 0x63C6A01: gst_buffer_pool_acquire_buffer
> > (gstbufferpool.c:1099)
> > ==8282== by 0xB8C663A: default_prepare_output_buffer
> > (gstbasetransform.c:1577)
> > ==8282== by 0xB8C83BA: gst_base_transform_handle_buffer
> > (gstbasetransform.c:2065)
> > ==8282== by 0xB8C8D00: gst_base_transform_chain
> > (gstbasetransform.c:2201)
> > ==8282==
> > ==8282== 221,634,126 bytes in 114 blocks are indirectly lost in loss
> > record 9,439 of 9,440
> > ==8282== at 0x4C2AB80: malloc (in
> > /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> > ==8282== by 0x4E85610: g_malloc (in /lib/x86_64-linux-gnu/libglib-
> > 2.0.so.0.4000.0)
> > ==8282== by 0x4E9B22D: g_slice_alloc (in /lib/x86_64-linux-
> > gnu/libglib-2.0.so.0.4000.0)
> > ==8282== by 0x63B87CC: _sysmem_new_block (gstallocator.c:415)
> > ==8282== by 0x63C2471: gst_buffer_new_allocate (gstbuffer.c:668)
> > ==8282== by 0x289F275C: video_buffer_pool_alloc
> (gstvideopool.c:195)
> > ==8282== by 0x63C5353: do_alloc_buffer.constprop.2
> > (gstbufferpool.c:272)
> > ==8282== by 0x63C56F5: default_acquire_buffer
> (gstbufferpool.c:995)
> > ==8282== by 0x63C6A01: gst_buffer_pool_acquire_buffer
> > (gstbufferpool.c:1099)
> > ==8282== by 0xB8C663A: default_prepare_output_buffer
> > (gstbasetransform.c:1577)
> > ==8282== by 0xB8C83BA: gst_base_transform_handle_buffer
> > (gstbasetransform.c:2065)
> > ==8282== by 0xB8C8D00: gst_base_transform_chain
> > (gstbasetransform.c:2201)
> > ==8282==
> > ==8282== 743,315,903 bytes in 377 blocks are still reachable in loss
> > record 9,440 of 9,440
> > ==8282== at 0x4C2AB80: malloc (in
> > /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> > ==8282== by 0x4E85610: g_malloc (in /lib/x86_64-linux-gnu/libglib-
> > 2.0.so.0.4000.0)
> > ==8282== by 0x4E9B22D: g_slice_alloc (in /lib/x86_64-linux-
> > gnu/libglib-2.0.so.0.4000.0)
> > ==8282== by 0x63B87CC: _sysmem_new_block (gstallocator.c:415)
> > ==8282== by 0x63C2471: gst_buffer_new_allocate (gstbuffer.c:668)
> > ==8282== by 0x289F275C: video_buffer_pool_alloc
> (gstvideopool.c:195)
> > ==8282== by 0x63C5353: do_alloc_buffer.constprop.2
> > (gstbufferpool.c:272)
> > ==8282== by 0x63C56F5: default_acquire_buffer
> (gstbufferpool.c:995)
> > ==8282== by 0x63C6A01: gst_buffer_pool_acquire_buffer
> > (gstbufferpool.c:1099)
> > ==8282== by 0xB8C663A: default_prepare_output_buffer
> > (gstbasetransform.c:1577)
> > ==8282== by 0xB8C83BA: gst_base_transform_handle_buffer
> > (gstbasetransform.c:2065)
> > ==8282== by 0xB8C8D00: gst_base_transform_chain
> > (gstbasetransform.c:2201)
> > ==8282==
> > ==8282== LEAK SUMMARY:
> > ==8282== definitely lost: 77,952 bytes in 274 blocks
> > ==8282== indirectly lost: 223,170,538 bytes in 879 blocks
> > ==8282== possibly lost: 175,129,187 bytes in 2,281 blocks
> > ==8282== still reachable: 772,523,997 bytes in 102,118 blocks
> > ==8282== suppressed: 0 bytes in 0 blocks
> > ==8282==
> > ==8282== For counts of detected and suppressed errors, rerun with: -v
> > ==8282== Use --track-origins=yes to see where uninitialised values
> come
> > from
> > ==8282== ERROR SUMMARY: 915 errors from 871 contexts (suppressed: 0
> > from 0)
> > (END)
> >
> >
> > Here a different run with G_SLICE:
> > ==13331== 190,527,582 bytes in 98 blocks are possibly lost in loss
> > record 9,466 of 9,468
> > ==13331== at 0x4C2AB80: malloc (in
> > /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> > ==13331== by 0x4E85610: g_malloc (in /lib/x86_64-linux-
> gnu/libglib-
> > 2.0.so.0.4000.0)
> > ==13331== by 0x4E9B22D: g_slice_alloc (in /lib/x86_64-linux-
> > gnu/libglib-2.0.so.0.4000.0)
> > ==13331== by 0x63B87CC: _sysmem_new_block (gstallocator.c:415)
> > ==13331== by 0x63C2471: gst_buffer_new_allocate (gstbuffer.c:668)
> > ==13331== by 0x287DD75C: video_buffer_pool_alloc
> > (gstvideopool.c:195)
> > ==13331== by 0x63C5353: do_alloc_buffer.constprop.2
> > (gstbufferpool.c:272)
> > ==13331== by 0x63C56F5: default_acquire_buffer
> (gstbufferpool.c:995)
> > ==13331== by 0x63C6A01: gst_buffer_pool_acquire_buffer
> > (gstbufferpool.c:1099)
> > ==13331== by 0xB8C663A: default_prepare_output_buffer
> > (gstbasetransform.c:1577)
> > ==13331== by 0xB8C83BA: gst_base_transform_handle_buffer
> > (gstbasetransform.c:2065)
> > ==13331== by 0xB8C8D00: gst_base_transform_chain
> > (gstbasetransform.c:2201)
> > ==13331==
> > ==13331== 217,745,808 bytes in 112 blocks are indirectly lost in loss
> > record 9,467 of 9,468
> > ==13331== at 0x4C2AB80: malloc (in
> > /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> > ==13331== by 0x4E85610: g_malloc (in /lib/x86_64-linux-
> gnu/libglib-
> > 2.0.so.0.4000.0)
> > ==13331== by 0x4E9B22D: g_slice_alloc (in /lib/x86_64-linux-
> > gnu/libglib-2.0.so.0.4000.0)
> > ==13331== by 0x63B87CC: _sysmem_new_block (gstallocator.c:415)
> > ==13331== by 0x63C2471: gst_buffer_new_allocate (gstbuffer.c:668)
> > ==13331== by 0x287DD75C: video_buffer_pool_alloc
> > (gstvideopool.c:195)
> > ==13331== by 0x63C5353: do_alloc_buffer.constprop.2
> > (gstbufferpool.c:272)
> > ==13331== by 0x63C56F5: default_acquire_buffer
> (gstbufferpool.c:995)
> > ==13331== by 0x63C6A01: gst_buffer_pool_acquire_buffer
> > (gstbufferpool.c:1099)
> > ==13331== by 0xB8C663A: default_prepare_output_buffer
> > (gstbasetransform.c:1577)
> > ==13331== by 0xB8C83BA: gst_base_transform_handle_buffer
> > (gstbasetransform.c:2065)
> > ==13331== by 0xB8C8D00: gst_base_transform_chain
> > (gstbasetransform.c:2201)
> > ==13331==
> > ==13331== 799,696,514 bytes in 406 blocks are still reachable in loss
> > record 9,468 of 9,468
> > ==13331== at 0x4C2AB80: malloc (in
> > /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> > ==13331== by 0x4E85610: g_malloc (in /lib/x86_64-linux-
> gnu/libglib-
> > 2.0.so.0.4000.0)
> > ==13331== by 0x4E9B22D: g_slice_alloc (in /lib/x86_64-linux-
> > gnu/libglib-2.0.so.0.4000.0)
> > ==13331== by 0x63B87CC: _sysmem_new_block (gstallocator.c:415)
> > ==13331== by 0x63C2471: gst_buffer_new_allocate (gstbuffer.c:668)
> > ==13331== by 0x287DD75C: video_buffer_pool_alloc
> > (gstvideopool.c:195)
> > ==13331== by 0x63C5353: do_alloc_buffer.constprop.2
> > (gstbufferpool.c:272)
> > ==13331== by 0x63C56F5: default_acquire_buffer
> (gstbufferpool.c:995)
> > ==13331== by 0x63C6A01: gst_buffer_pool_acquire_buffer
> > (gstbufferpool.c:1099)
> > ==13331== by 0xB8C663A: default_prepare_output_buffer
> > (gstbasetransform.c:1577)
> > ==13331== by 0xB8C83BA: gst_base_transform_handle_buffer
> > (gstbasetransform.c:2065)
> > ==13331== by 0xB8C8D00: gst_base_transform_chain
> > (gstbasetransform.c:2201)
> > ==13331==
> > ==13331== LEAK SUMMARY:
> > ==13331== definitely lost: 78,872 bytes in 277 blocks
> > ==13331== indirectly lost: 221,424,134 bytes in 906 blocks
> > ==13331== possibly lost: 190,732,025 bytes in 2,769 blocks
> > ==13331== still reachable: 841,250,822 bytes in 102,333 blocks
> > ==13331== suppressed: 0 bytes in 0 blocks
> > ==13331==
> > ==13331== For counts of detected and suppressed errors, rerun with: -
> v
> > ==13331== Use --track-origins=yes to see where uninitialised values
> > come from
> > ==13331== ERROR SUMMARY: 1098 errors from 874 contexts (suppressed: 0
> > from 0)
> >
> >
> > --
> > Best regards,
> > Krzysztof Borowczyk
> >
> > _______________________________________________
> > gstreamer-devel mailing list
> > gstreamer-devel at lists.freedesktop.org
> > http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
More information about the gstreamer-devel
mailing list