Poor HD performance with appsink

Thu Feb 13 09:42:35 PST 2014

I am capturing OpenGL output and pushing it to appsrc. OpenGL is drawing 
videos which are from appsink. What I want to say is you must be 
carefully about copying buffers around because it can cause huge slow downs.

Dňa 13.02.2014 18:36, Gordon Nickel wrote / napísal(a):
> I'm confused - I thought appsrc was to send video data TO gstreamer and appsink was to get data FROM gstreamer. How would I display video with appsrc?
>
> -----Original Message-----
> From: gstreamer-devel-bounces at lists.freedesktop.org [mailto:gstreamer-devel-bounces at lists.freedesktop.org] On Behalf Of Dušan Poizl
> Sent: Thursday, February 13, 2014 9:35 AM
> To: Discussion of the development of and with GStreamer
> Subject: Re: Poor HD performance with appsink
>
> Well in my case there was more gain in appsrc. Before I could barely capture (often it dropped below 20fps) and encode 720p video while it played single video 720p. When I removed unnecessary copying in both appsink and appsrc it bring tremendous improvment. Now I can capture and encode full hd video while playing two full hd video and there is still some CPU left.
>
> Dňa 13.02.2014 18:22, Gordon Nickel wrote / napísal(a):
>> Ah - you're also using appsrc, my issue is with appsink.
>>
>> Though the copying straight to / from a surface could be a speed improvement. Mine has an intermediate copy step.
>>
>>
>> -----Original Message-----
>> From: gstreamer-devel-bounces at lists.freedesktop.org
>> [mailto:gstreamer-devel-bounces at lists.freedesktop.org] On Behalf Of
>> Dušan Poizl
>> Sent: Thursday, February 13, 2014 9:13 AM
>> To: Discussion of the development of and with GStreamer
>> Subject: Re: Poor HD performance with appsink
>>
>> Yea I had similiar problem with capturing the OpenGL output at there was three copy of data and unnecessary RGB -> BGRA conversion (glReadPixels is fastest with GL_BGRA) before it was pushed to appsrc.
>>
>> my code is in newSample callback
>>
>> void *data = videoSurface->lockFrame();//lock the memory so renderer
>> doesn't access this memory gsize size = gst_buffer_get_size(buffer);
>> gst_buffer_extract(buffer, 0, data, size);//extract directly to my
>> memory
>> videoSurface->unlockFrame();//unlock memory and signal to redraw
>>
>> in draw method I call where data is pointer which returned by
>> lockFrame()
>>
>> for(uint i=0;i<GST_VIDEO_INFO_N_PLANES(&videoInfo);i++)
>>                {
>>                    glActiveTexture(GL_TEXTURE0+i);
>>                    glBindTexture(GL_TEXTURE_2D, textures[i]);
>>                glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0,
>> GST_VIDEO_INFO_COMP_WIDTH(&videoInfo, i),
>> GST_VIDEO_INFO_COMP_HEIGHT(&videoInfo, i), GL_RED, GL_UNSIGNED_BYTE,
>> data+GST_VIDEO_INFO_PLANE_OFFSET(&videoInfo, i));
>> }
>>
>> Dňa 13.02.2014 17:57, Gordon Nickel wrote / napísal(a):
>>> Thanks Dusan, that's very helpful.
>>>
>>> How are you getting the video data from Gstreamer to OpenGL? I suspect that might be where the slowdown might be happening in my code, since the copy is on the main thread.
>>>
>>> Here's the implementation I'm using:
>>> https://gist.github.com/gordeaoux/8978998
>>>
>>> Which is based on this cinder wrapper:
>>> https://github.com/cadet/_2RealGStreamerWrapper/blob/master/src/_2Rea
>>> lGStreamerWrapper.cpp
>>>
>>> Gordon Nickel
>>>
>>> -----Original Message-----
>>> From: gstreamer-devel-bounces at lists.freedesktop.org
>>> [mailto:gstreamer-devel-bounces at lists.freedesktop.org] On Behalf Of
>>> Dušan Poizl
>>> Sent: Thursday, February 13, 2014 8:26 AM
>>> To: Discussion of the development of and with GStreamer
>>> Subject: Re: Poor HD performance with appsink
>>>
>>> I used QtGstreamer but because it is not for gstreamer-1.0 I must
>>> rewrite my SW to use gstreamer directly. But you can get general idea
>>> from this code
>>> http://cgit.freedesktop.org/gstreamer/qt-gstreamer/tree/elements/gstq
>>> tvideosink/painters/openglsurfacepainter.cpp
>>> which is where I started when I wrote my my own implementation.
>>>
>>> Dňa 13.02.2014 16:45, Gordon Nickel wrote / napísal(a):
>>>> Thanks for the responses. I'm using callbacks, not signals already.
>>>>
>>>> Dusan, how are you displaying content in OpenGL? Is there an implementation you would be willing to share?
>>>>
>>>> Gordon Nickel
>>>>
>>>> downstream.com
>>>>
>>>>> On Feb 13, 2014, at 7:39 AM, "Dušan Poizl" <poizl at maindata.sk> wrote:
>>>>>
>>>>> I am using appsink for playing video with OpenGL and have no issue. I can easily play two FullHD video fine while capturing output and pushing to appsrc to encode.
>>>>>
>>>>> Dňa 13.02.2014 16:14, Aleix Conchillo Flaqué wrote / napísal(a):
>>>>>> On Wed, Feb 12, 2014 at 5:40 PM, Gordon Nickel
>>>>>> <gordon.nickel at downstream.com> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I have an implementation of Gstreamer 1.0 for Windows using
>>>>>>> appsink to play videos back. Generally, performance is pretty
>>>>>>> good, and quality is excellent. However, with some specific
>>>>>>> videos (all 1080p, 720p seems to be fine), the video track can
>>>>>>> get very far behind the audio, at which time the audio starts to
>>>>>>> stutter and eventually cut out. I don't believe it's an issue
>>>>>>> with the app itself (which generally gets 90-200 fps with videos
>>>>>>> running), but my implementation of Gstreamer. The same videos play without issue using playbin in gst-launch.
>>>>>>>
>>>>>>> I've tried setting drop frames ( gst_app_sink_set_drop to true )
>>>>>>> and max lateness to various values from 1 second to 0.01 seconds.
>>>>>>> In doing so, the audio won't cut out, but the video will stutter
>>>>>>> to an unnacceptable degree (approx 1 frame per second). I've also
>>>>>>> tried different bit rates, but have generally stuck with h.264
>>>>>>> for codecs. Mpeg 2 had similar performance issues.
>>>>>>>
>>>>>>> Playing the video through the app but with it's own
>>>>>>> directdrawsink window played back smoothly.
>>>>>>>
>>>>>>> I can provide the problem videos if that would be helpful.
>>>>>>>
>>>>>>> Gstreamer builds 1.0.10, 1.2.0 and 1.2.3 have all been tried with
>>>>>>> the same results.
>>>>>>>
>>>>>>> Any ideas to improve performance?
>>>>>> You might already be doing it, but do you use signals instead of
>>>>>> callbacks? If so, use callbacks instead. Much faster.
>>>>>>
>>>>>> Aleix
>>>>>> _______________________________________________
>>>>>> gstreamer-devel mailing list
>>>>>> gstreamer-devel at lists.freedesktop.org
>>>>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>>>> _______________________________________________
>>>>> gstreamer-devel mailing list
>>>>> gstreamer-devel at lists.freedesktop.org
>>>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>>> _______________________________________________
>>>> gstreamer-devel mailing list
>>>> gstreamer-devel at lists.freedesktop.org
>>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>> gstreamer-devel at lists.freedesktop.org
>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>> gstreamer-devel at lists.freedesktop.org
>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel