Webrtcbin: Huge increasing memory leak while using appsink instead of autovideosink

[Neil Young]

Hi,

I was reporting this as an issue here https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1010 <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1010>, but was redirected to this list. Not sure if it is the right place, but I will try.

I'm referring to the Java sample app: https://github.com/centricular/gstwebrtc-demos/tree/master/sendrecv/gst-java <https://github.com/centricular/gstwebrtc-demos/tree/master/sendrecv/gst-java>
With respect to increasing memory consumption everything is fine and balanced if we are using this approach to display the received video in a pop-up window:

if (name.startsWith("video")) {
            Element queue = ElementFactory.make("queue", "my-videoqueue");
            Element videoconvert = ElementFactory.make("videoconvert", "my-videoconvert");
            Element autovideosink = ElementFactory.make("osxvideosink", "my-autovideosink");
//            Element autovideosink = ElementFactory.make("autovideosink", "my-autovideosink");
            pipe.addMany(queue, videoconvert, autovideosink);
            queue.syncStateWithParent();
            videoconvert.syncStateWithParent();
            autovideosink.syncStateWithParent();
            pad.link(queue.getStaticPad("sink"));
            queue.link(videoconvert);
            videoconvert.link(autovideosink);
}
(autovideosink is replaced by osxvideosink for macOS)

If this sequence above is replaced by the following, the memory consumption of the Java app grows by 200 MB/s, which after a while leads to a complete crash of the app if not of the entire system:

if (name.startsWith("video")) {
            Element queue = ElementFactory.make("queue", "my-videoqueue");
            Element videoconvert = ElementFactory.make("videoconvert", "my-videoconvert");
            AppSink sink = (AppSink) ElementFactory.make("appsink", "my-appsink");
            sink.set("emit-signals", true);
            sink.connect(new AppSink.NEW_SAMPLE() {
                @Override
                public FlowReturn newSample(AppSink elem) {
                    Sample sample = elem.pullSample();
                    Structure capsStruct = sample.getCaps().getStructure(0);
                    String format = capsStruct.getString("format");
                    int width = capsStruct.getInteger("width");
                    int height = capsStruct.getInteger("height");

                    ByteBuffer bytes = sample.getBuffer().map(false);
                    byte[] buffer = new byte[bytes.remaining()];
                    bytes.get(buffer);
                    dragonfly.onNextFrame(format, buffer, width, height);

                    sample.dispose();
                    return FlowReturn.OK;
                }
            });

            sink.connect(new AppSink.NEW_PREROLL() {
                @Override
                public FlowReturn newPreroll(AppSink elem) {
                    Sample sample = elem.pullPreroll();
                    Structure capsStruct = sample.getCaps().getStructure(0);
                    String format = capsStruct.getString("format");

                    int width = capsStruct.getInteger("width");
                    int height = capsStruct.getInteger("height");

                    ByteBuffer bytes = sample.getBuffer().map(false);
                    byte[] buffer = new byte[bytes.remaining()];
                    bytes.get(buffer);
                    dragonfly.onNextFrame(format, buffer, width, height);

                    sample.dispose();
                    return FlowReturn.OK;
                }
            });

            pipe.addMany(queue, videoconvert, sink);
            queue.syncStateWithParent();

            videoconvert.syncStateWithParent();
            sink.syncStateWithParent();
            pad.link(queue.getStaticPad("sink"));
            queue.link(videoconvert);
            videoconvert.link(sink);
}
Is there anything wrong with the sequence above, which could cause the leak? The copy of the buffers (instead of forwarding) is necessary, since the final handling is done in native code, not under my control. And even if we don't do anything in the callbacks - the simple replace of autovideosink vs appsink causes the leak.



TIA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20190708/d9300e27/attachment-0001.html>