AW: AW: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

Thu Apr 30 11:41:53 UTC 2020

Hi all,

after further tests we decided to open an issue:

https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/issues/253

There seems to be a bug in the DMABuf-Handling.

The issue #253 also contains a small application to demonstrate the problem.

Best regards
Michael

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:gstreamer-devel-bounces at lists.freedesktop.org] Im Auftrag von Timtchenko, Michael
Gesendet: Donnerstag, 23. April 2020 10:28
An: Discussion of the development of and with GStreamer <gstreamer-devel at lists.freedesktop.org>; Junyan He <junyan.he at hotmail.com>
Betreff: AW: AW: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

Or maybe an example how to convert a VASurface to a GLTexture. This would be very helpful.

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:gstreamer-devel-bounces at lists.freedesktop.org] Im Auftrag von Timtchenko, Michael
Gesendet: Donnerstag, 23. April 2020 10:04
An: Discussion of the development of and with GStreamer <gstreamer-devel at lists.freedesktop.org>; Junyan He <junyan.he at hotmail.com>
Betreff: AW: AW: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

Hi Junyan,

we'll try to integrate the merge-requests and evaluate the issue again. I'll share the results.

I would like to try the VASurface variant. Do you have some example code, how to transform a eglimage to a gltexture? If so, I could easily provide some feedback. Maybe this would lead to further isolation of the problem source.

BR,
Michael

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:gstreamer-devel-bounces at lists.freedesktop.org] Im Auftrag von Junyan He
Gesendet: Donnerstag, 23. April 2020 09:04
An: Timtchenko, Michael <Michael.Timtchenko at agcocorp.com>; Discussion of the development of and with GStreamer <gstreamer-devel at lists.freedesktop.org>
Betreff: Re: AW: AW: [External] Re: vaapipostproc + (brightness|saturation|contrast) + huge memory leak

If KMS sink, the vaapipostproc should copy the data back to sys memory, not using DMA or VA surface.

If that does not leak memory, it seems that the internal logic of postproc or decoder may not leak memory.

The DMA buffer logic really has problems, and we already do something to fix it.

https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/251

https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/276

https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/303

but they are not included in 1.16.2

On 2020年04月23日 14:50, Timtchenko, Michael wrote:
> Hi Junyan,
>
> thx 4 ur reply!
>
> I'm working on an EGL-System without a window manager. Therefore a GLX
> test is not possible. (It's an embedded device)
>
> Nonetheless, I tried already "video/x-raw(memory:VASurface)" but had no success in transforming the EGLImage to a GL-Texture manually. That’s why I'm using glupload to achieve this goal.
> Do you think, that using the VASurface instead, would solve the problem?
>
> I'm quite sure that there is an issue. Without changing the mentioned properties, the whole pipeline works very nice and efficient.
>
> Maybe this is helpful for further analysis:
>
> When trying to run the pipeline, without "video/x-raw(memory:DMABuf)" and streaming in an kmlsink, there's no increase of memory at all (of course in the same scenario with changing property values etc.). In other words, the following pipeline allows property manipulation without memory issues:
>
> udpsrc port=55555 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96"
> ! rtpjitterbuffer latency=20
> ! rtph264depay
> ! vaapih264dec
> ! vaapipostproc
> ! kmssink
>
> So I guess you are right, that there is sth wrong related to the DMABuf code.
>
> Any further advice?
>
> BR
> Michael
>
>
>
>
> -----Ursprüngliche Nachricht-----
> Von: Junyan He [mailto:junyan.he at hotmail.com]
> Gesendet: Mittwoch, 22. April 2020 17:10
> An: Discussion of the development of and with GStreamer
> <gstreamer-devel at lists.freedesktop.org>; Timtchenko, Michael
> <Michael.Timtchenko at agcocorp.com>
> Betreff: Re: AW: [External] Re: vaapipostproc +
> (brightness|saturation|contrast) + huge memory leak
>
> Do you use DMB buffer and EGL?
>
> Can you try to use VASurface and GLX to have a test?
>
> We really have latent issue in EGL's DMA buffer.
>
>
>
> On 2020年04月22日 22:18, Timtchenko, Michael wrote:
>> Hi Victor,
>>
>> thanks for your reply. Unfortunately I do not have a small executable
>> to demonstrate my problem. (Maybe I can provide it later)
>>
>> Further information:
>>
>> - I don't change the state of the pipeline very often, maybe 2-3 times. Nevertheless, the mentioned effect in the github issue seems to be similar to my problem.
>>
>> - I use vaapih264dec as a decoder
>>
>> - I detect the leak by use of "free" and "htop" with my target
>> process (there is a significant increase of memory usage while
>> changing
>> calibration)
>> - There's only code for changing the calibration, nothing else. That's why I think, that the leak must be within the postprocessing. Running application without color calibration does not cause any increase of memory usage.
>>
>> My full pipeline looks like this:
>>
>> auto pipeline = gst_pipeline_new( "pipeline" );
>>
>> auto src = gst_element_factory_make( "udpsrc", "source" ); auto depay
>> = gst_element_factory_make( "rtph264depay", "depay" ); auto dec =
>> gst_element_factory_make( "vaapih264dec", "dec" ); auto postproc =
>> gst_element_factory_make( "vaapipostproc", "postproc" ); auto upload
>> = gst_element_factory_make( "glupload", "upload" ); auto sink =
>> gst_element_factory_make( "appsink", "sink" );
>>
>> gst_bin_add_many( GST_BIN(pipeline), src, depay, dec, postproc,
>> upload, sink, NULL );
>>
>> if( !gst_element_link_many( src, depay, dec, postproc, NULL ) ) {
>> return -1; }
>>
>> GstCaps* caps = gst_caps_from_string( "video/x-raw(memory:DMABuf),
>> format=RGBA" );
>>
>> if( !gst_element_link_filtered( postproc, upload, caps ) ) { return
>> -1; }
>>
>> if( !gst_element_link( upload, sink ) ) { return -1; }
>>
>> // Setup Caps, Ports of src
>>
>> // Inject GlContext
>>
>> // Register Callbacks etc.
>>
>> Thanks in advance!
>>
>> BR
>> Michael
>>
>> -----Ursprüngliche Nachricht-----
>> Von: gstreamer-devel
>> [mailto:gstreamer-devel-bounces at lists.freedesktop.org] Im Auftrag von
>> Víctor Jáquez
>> Gesendet: Mittwoch, 22. April 2020 08:38
>> An: Discussion of the development of and with GStreamer
>> <gstreamer-devel at lists.freedesktop.org>
>> Betreff: [External] Re: vaapipostproc +
>> (brightness|saturation|contrast) + huge memory leak
>>
>> On Tue, 21 Apr 2020 at 12:55, Timtchenko, Michael wrote:
>>> Hi all,
>>> i'm currently facing a very strange behaviour within the gstreamer
>>> module vaapipostproc in version 1.16.2. I'm using this module for
>>> postprocessing of my video stream. Especially for adjustment of
>>> brightness, saturation and contrast, by changing the modules properties.
>>> The decoding of the stream with vaapi, as well as the
>>> gl-texture-upload by glupload works like a charm. But when the
>>> stream is running and I'm trying to change the properties for
>>> brightness, saturation or contrast many times (in a calibration manner), I'm running out of memory.
>>> Is it possible that there is a huge memory leak? When I'm steadily
>>> calibrating the mentioned properties for about 30 secs, I get an
>>> increase of memory about
>>> 2 GB. (I know from 14.4.4, that there were some issues related to a
>>> mutual exclusive color calibration, which were fixed by a patch.)
>>> Could someone verify this issue?
>>> Below I've attached the relevant code sequences:
>>> Construction:
>>> ...
>>>       auto postproc = gst_element_factory_make( "vaapipostproc", "postproc" );
>>>       auto upload = gst_element_factory_make( "glupload", "upload" );
>>>       GstCaps* caps = gst_caps_from_string( "video/x-raw(memory:DMABuf), format=RGBA" );
>>>       if( !gst_element_link_filtered( postproc, upload, caps ) ) ...
>>> Setter for Properties (the lines below are executed very often, while stream is running):
>>> ...
>>> gfloat tmp = 0.5
>>> g_object_set( G_OBJECT( vaapipostproc ), "brightness", tmp, NULL );
>>> ...
>>> Not sure whether i'm doing wrong, or there is sth broken in vaapipostproc.
>>> Kind regards
>>> Michael
>> Hi Michael,
>>
>> Are you changing the state of the pipeline frequently? I ask it because there's a bug, but only reported on decoders:
>> https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requ
>> e
>> sts/27
>>
>> How do you measure the memory leak?
>>
>> You mention "gl-texture-upload" with decoders, but your pipeline uses dmabuf. Which one are you using completely? Do you have a simple test program to replicate the leak?
>>
>> cheers
>>
>> vmjl
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
>>
>> Bitte beachten / Please note!
>>
>> *********************************************************************
>> *
>> *********
>>
>> AGCO GmbH
>> Sitz der AGCO GmbH: Johann-Georg-Fendt-Str.4, 87616 Marktoberdorf,
>> Germany Registergericht Amtsgericht Kempten HRB 10327
>> Geschäftsführer: Christoph Groeblinghoff, Ingrid Bussjaeger-Martin,
>> Dr. Heribert Reiter, Ekkehart Glaeser Vorsitzender des Aufsichtsrates:
>> Torsten Dehner
>>
>> *********************************************************************
>> *
>> *********
>>
>> Diese E-Mail ist nur für den Empfänger bestimmt, an den es gerichtet
>> ist und kann vertrauliches bzw. unter das Berufsgeheimnis fallendes
>> Material enthalten. Jegliche darin enthaltene Ansicht oder
>> Meinungsäußerung ist die des Autors und stellt nicht notwendigerweise
>> die Ansicht oder Meinung von AGCO dar. Sind Sie nicht der Empfänger,
>> so haben Sie diese E-Mail irrtümlich erhalten und jegliche
>> Verwendung, Veröffentlichung, Weiterleitung, Abschrift oder jeglicher
>> Druck dieser E-Mail ist strengstens untersagt. Weder AGCO noch der Absender übernehmen die Haftung für Viren; es obliegt Ihrer Verantwortung, die E-Mail und deren angehängte Dateien (sofern vorhanden) auf Viren zu überprüfen.
>>
>> *********************************************************************
>> *
>> *********
>>
>> This email is intended solely for the use of the individual to whom
>> it is addressed and may contain confidential and/or privileged material.
>> Any views or opinions presented are solely those of the author and do
>> not necessarily represent those of AGCO. If you are not the intended
>> recipient, be advised that you have received this email in error and
>> that any use, dissemination, forwarding, printing or copying of this
>> email is strictly prohibited. Neither AGCO nor the sender accepts any
>> responsibility for viruses and it is your responsibility to scan and
>> virus check the email and its
>> attachment(s) (if any).
>>
>> *********************************************************************
>> *
>> ********* _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> This email is intended solely for the use of the individual to whom it is addressed and may contain confidential and/or privileged material. Any views or opinions presented are solely those of the author and do not necessarily represent those of AGCO. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any responsibility for viruses and it is your responsibility to scan and virus check the e-mail and its attachment(s) (if any).

_______________________________________________
gstreamer-devel mailing list
gstreamer-devel at lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

This email is intended solely for the use of the individual to whom it is addressed and may contain confidential and/or privileged material. Any views or opinions presented are solely those of the author and do not necessarily represent those of AGCO. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any responsibility for viruses and it is your responsibility to scan and virus check the e-mail and its attachment(s) (if any).
_______________________________________________
gstreamer-devel mailing list
gstreamer-devel at lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

This email is intended solely for the use of the individual to whom it is addressed and may contain confidential and/or privileged material. Any views or opinions presented are solely those of the author and do not necessarily represent those of AGCO. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any responsibility for viruses and it is your responsibility to scan and virus check the e-mail and its attachment(s) (if any).