Poor HD performance with appsink

Gordon Nickel gordon.nickel at downstream.com
Thu Feb 13 09:36:37 PST 2014 

I'm confused - I thought appsrc was to send video data TO gstreamer and appsink was to get data FROM gstreamer. How would I display video with appsrc?

-----Original Message-----
From: gstreamer-devel-bounces at lists.freedesktop.org [mailto:gstreamer-devel-bounces at lists.freedesktop.org] On Behalf Of Dušan Poizl
Sent: Thursday, February 13, 2014 9:35 AM
To: Discussion of the development of and with GStreamer
Subject: Re: Poor HD performance with appsink

Well in my case there was more gain in appsrc. Before I could barely capture (often it dropped below 20fps) and encode 720p video while it played single video 720p. When I removed unnecessary copying in both appsink and appsrc it bring tremendous improvment. Now I can capture and encode full hd video while playing two full hd video and there is still some CPU left.

Dňa 13.02.2014 18:22, Gordon Nickel wrote / napísal(a):
> Ah - you're also using appsrc, my issue is with appsink.
>
> Though the copying straight to / from a surface could be a speed improvement. Mine has an intermediate copy step.
>
>
> -----Original Message-----
> From: gstreamer-devel-bounces at lists.freedesktop.org 
> [mailto:gstreamer-devel-bounces at lists.freedesktop.org] On Behalf Of 
> Dušan Poizl
> Sent: Thursday, February 13, 2014 9:13 AM
> To: Discussion of the development of and with GStreamer
> Subject: Re: Poor HD performance with appsink
>
> Yea I had similiar problem with capturing the OpenGL output at there was three copy of data and unnecessary RGB -> BGRA conversion (glReadPixels is fastest with GL_BGRA) before it was pushed to appsrc.
>
> my code is in newSample callback
>
> void *data = videoSurface->lockFrame();//lock the memory so renderer 
> doesn't access this memory gsize size = gst_buffer_get_size(buffer); 
> gst_buffer_extract(buffer, 0, data, size);//extract directly to my 
> memory
> videoSurface->unlockFrame();//unlock memory and signal to redraw
>
> in draw method I call where data is pointer which returned by 
> lockFrame()
>
> for(uint i=0;i<GST_VIDEO_INFO_N_PLANES(&videoInfo);i++)
>               {
>                   glActiveTexture(GL_TEXTURE0+i);
>                   glBindTexture(GL_TEXTURE_2D, textures[i]);
>               glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 
> GST_VIDEO_INFO_COMP_WIDTH(&videoInfo, i), 
> GST_VIDEO_INFO_COMP_HEIGHT(&videoInfo, i), GL_RED, GL_UNSIGNED_BYTE,
> data+GST_VIDEO_INFO_PLANE_OFFSET(&videoInfo, i));
> }
>
> Dňa 13.02.2014 17:57, Gordon Nickel wrote / napísal(a):
>> Thanks Dusan, that's very helpful.
>>
>> How are you getting the video data from Gstreamer to OpenGL? I suspect that might be where the slowdown might be happening in my code, since the copy is on the main thread.
>>
>> Here's the implementation I'm using: 
>> https://gist.github.com/gordeaoux/8978998
>>
>> Which is based on this cinder wrapper: 
>> https://github.com/cadet/_2RealGStreamerWrapper/blob/master/src/_2Rea
>> lGStreamerWrapper.cpp
>>
>> Gordon Nickel
>>
>> -----Original Message-----
>> From: gstreamer-devel-bounces at lists.freedesktop.org 
>> [mailto:gstreamer-devel-bounces at lists.freedesktop.org] On Behalf Of 
>> Dušan Poizl
>> Sent: Thursday, February 13, 2014 8:26 AM
>> To: Discussion of the development of and with GStreamer
>> Subject: Re: Poor HD performance with appsink
>>
>> I used QtGstreamer but because it is not for gstreamer-1.0 I must 
>> rewrite my SW to use gstreamer directly. But you can get general idea 
>> from this code 
>> http://cgit.freedesktop.org/gstreamer/qt-gstreamer/tree/elements/gstq
>> tvideosink/painters/openglsurfacepainter.cpp
>> which is where I started when I wrote my my own implementation.
>>
>> Dňa 13.02.2014 16:45, Gordon Nickel wrote / napísal(a):
>>> Thanks for the responses. I'm using callbacks, not signals already.
>>>
>>> Dusan, how are you displaying content in OpenGL? Is there an implementation you would be willing to share?
>>>
>>> Gordon Nickel
>>>
>>> downstream.com
>>>
>>>> On Feb 13, 2014, at 7:39 AM, "Dušan Poizl" <poizl at maindata.sk> wrote:
>>>>
>>>> I am using appsink for playing video with OpenGL and have no issue. I can easily play two FullHD video fine while capturing output and pushing to appsrc to encode.
>>>>
>>>> Dňa 13.02.2014 16:14, Aleix Conchillo Flaqué wrote / napísal(a):
>>>>> On Wed, Feb 12, 2014 at 5:40 PM, Gordon Nickel 
>>>>> <gordon.nickel at downstream.com> wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I have an implementation of Gstreamer 1.0 for Windows using 
>>>>>> appsink to play videos back. Generally, performance is pretty 
>>>>>> good, and quality is excellent. However, with some specific 
>>>>>> videos (all 1080p, 720p seems to be fine), the video track can 
>>>>>> get very far behind the audio, at which time the audio starts to 
>>>>>> stutter and eventually cut out. I don't believe it's an issue 
>>>>>> with the app itself (which generally gets 90-200 fps with videos 
>>>>>> running), but my implementation of Gstreamer. The same videos play without issue using playbin in gst-launch.
>>>>>>
>>>>>> I've tried setting drop frames ( gst_app_sink_set_drop to true ) 
>>>>>> and max lateness to various values from 1 second to 0.01 seconds.
>>>>>> In doing so, the audio won't cut out, but the video will stutter 
>>>>>> to an unnacceptable degree (approx 1 frame per second). I've also 
>>>>>> tried different bit rates, but have generally stuck with h.264 
>>>>>> for codecs. Mpeg 2 had similar performance issues.
>>>>>>
>>>>>> Playing the video through the app but with it's own 
>>>>>> directdrawsink window played back smoothly.
>>>>>>
>>>>>> I can provide the problem videos if that would be helpful.
>>>>>>
>>>>>> Gstreamer builds 1.0.10, 1.2.0 and 1.2.3 have all been tried with 
>>>>>> the same results.
>>>>>>
>>>>>> Any ideas to improve performance?
>>>>> You might already be doing it, but do you use signals instead of 
>>>>> callbacks? If so, use callbacks instead. Much faster.
>>>>>
>>>>> Aleix
>>>>> _______________________________________________
>>>>> gstreamer-devel mailing list
>>>>> gstreamer-devel at lists.freedesktop.org
>>>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>>> _______________________________________________
>>>> gstreamer-devel mailing list
>>>> gstreamer-devel at lists.freedesktop.org
>>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>> gstreamer-devel at lists.freedesktop.org
>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
gstreamer-devel at lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

More information about the gstreamer-devel mailing list