Poor HD performance with appsink

[Dušan Poizl]

Well in my case there was more gain in appsrc. Before I could barely 
capture (often it dropped below 20fps) and encode 720p video while it 
played single video 720p. When I removed unnecessary copying in both 
appsink and appsrc it bring tremendous improvment. Now I can capture and 
encode full hd video while playing two full hd video and there is still 
some CPU left.

Dňa 13.02.2014 18:22, Gordon Nickel wrote / napísal(a):
> Ah - you're also using appsrc, my issue is with appsink.
>
> Though the copying straight to / from a surface could be a speed improvement. Mine has an intermediate copy step.
>
>
> -----Original Message-----
> From: gstreamer-devel-bounces at lists.freedesktop.org [mailto:gstreamer-devel-bounces at lists.freedesktop.org] On Behalf Of Dušan Poizl
> Sent: Thursday, February 13, 2014 9:13 AM
> To: Discussion of the development of and with GStreamer
> Subject: Re: Poor HD performance with appsink
>
> Yea I had similiar problem with capturing the OpenGL output at there was three copy of data and unnecessary RGB -> BGRA conversion (glReadPixels is fastest with GL_BGRA) before it was pushed to appsrc.
>
> my code is in newSample callback
>
> void *data = videoSurface->lockFrame();//lock the memory so renderer doesn't access this memory gsize size = gst_buffer_get_size(buffer); gst_buffer_extract(buffer, 0, data, size);//extract directly to my memory
> videoSurface->unlockFrame();//unlock memory and signal to redraw
>
> in draw method I call where data is pointer which returned by lockFrame()
>
> for(uint i=0;i<GST_VIDEO_INFO_N_PLANES(&videoInfo);i++)
>               {
>                   glActiveTexture(GL_TEXTURE0+i);
>                   glBindTexture(GL_TEXTURE_2D, textures[i]);
>               glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0,
> GST_VIDEO_INFO_COMP_WIDTH(&videoInfo, i),
> GST_VIDEO_INFO_COMP_HEIGHT(&videoInfo, i), GL_RED, GL_UNSIGNED_BYTE,
> data+GST_VIDEO_INFO_PLANE_OFFSET(&videoInfo, i));
> }
>
> Dňa 13.02.2014 17:57, Gordon Nickel wrote / napísal(a):
>> Thanks Dusan, that's very helpful.
>>
>> How are you getting the video data from Gstreamer to OpenGL? I suspect that might be where the slowdown might be happening in my code, since the copy is on the main thread.
>>
>> Here's the implementation I'm using: https://gist.github.com/gordeaoux/8978998
>>
>> Which is based on this cinder wrapper: https://github.com/cadet/_2RealGStreamerWrapper/blob/master/src/_2RealGStreamerWrapper.cpp
>>
>> Gordon Nickel
>>
>> -----Original Message-----
>> From: gstreamer-devel-bounces at lists.freedesktop.org [mailto:gstreamer-devel-bounces at lists.freedesktop.org] On Behalf Of Dušan Poizl
>> Sent: Thursday, February 13, 2014 8:26 AM
>> To: Discussion of the development of and with GStreamer
>> Subject: Re: Poor HD performance with appsink
>>
>> I used QtGstreamer but because it is not for gstreamer-1.0 I must rewrite my SW to use gstreamer directly. But you can get general idea from this code http://cgit.freedesktop.org/gstreamer/qt-gstreamer/tree/elements/gstqtvideosink/painters/openglsurfacepainter.cpp
>> which is where I started when I wrote my my own implementation.
>>
>> Dňa 13.02.2014 16:45, Gordon Nickel wrote / napísal(a):
>>> Thanks for the responses. I'm using callbacks, not signals already.
>>>
>>> Dusan, how are you displaying content in OpenGL? Is there an implementation you would be willing to share?
>>>
>>> Gordon Nickel
>>>
>>> downstream.com
>>>
>>>> On Feb 13, 2014, at 7:39 AM, "Dušan Poizl" <poizl at maindata.sk> wrote:
>>>>
>>>> I am using appsink for playing video with OpenGL and have no issue. I can easily play two FullHD video fine while capturing output and pushing to appsrc to encode.
>>>>
>>>> Dňa 13.02.2014 16:14, Aleix Conchillo Flaqué wrote / napísal(a):
>>>>> On Wed, Feb 12, 2014 at 5:40 PM, Gordon Nickel
>>>>> <gordon.nickel at downstream.com> wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I have an implementation of Gstreamer 1.0 for Windows using appsink
>>>>>> to play videos back. Generally, performance is pretty good, and
>>>>>> quality is excellent. However, with some specific videos (all
>>>>>> 1080p, 720p seems to be fine), the video track can get very far
>>>>>> behind the audio, at which time the audio starts to stutter and
>>>>>> eventually cut out. I don't believe it's an issue with the app
>>>>>> itself (which generally gets 90-200 fps with videos running), but
>>>>>> my implementation of Gstreamer. The same videos play without issue using playbin in gst-launch.
>>>>>>
>>>>>> I've tried setting drop frames ( gst_app_sink_set_drop to true )
>>>>>> and max lateness to various values from 1 second to 0.01 seconds.
>>>>>> In doing so, the audio won't cut out, but the video will stutter to
>>>>>> an unnacceptable degree (approx 1 frame per second). I've also
>>>>>> tried different bit rates, but have generally stuck with h.264 for
>>>>>> codecs. Mpeg 2 had similar performance issues.
>>>>>>
>>>>>> Playing the video through the app but with it's own directdrawsink
>>>>>> window played back smoothly.
>>>>>>
>>>>>> I can provide the problem videos if that would be helpful.
>>>>>>
>>>>>> Gstreamer builds 1.0.10, 1.2.0 and 1.2.3 have all been tried with
>>>>>> the same results.
>>>>>>
>>>>>> Any ideas to improve performance?
>>>>> You might already be doing it, but do you use signals instead of
>>>>> callbacks? If so, use callbacks instead. Much faster.
>>>>>
>>>>> Aleix
>>>>> _______________________________________________
>>>>> gstreamer-devel mailing list
>>>>> gstreamer-devel at lists.freedesktop.org
>>>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>>> _______________________________________________
>>>> gstreamer-devel mailing list
>>>> gstreamer-devel at lists.freedesktop.org
>>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>> gstreamer-devel at lists.freedesktop.org
>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel