Poor HD performance with appsink

[Dušan Poizl]

Yea I had similiar problem with capturing the OpenGL output at there was 
three copy of data and unnecessary RGB -> BGRA conversion (glReadPixels 
is fastest with GL_BGRA) before it was pushed to appsrc.

my code is in newSample callback

void *data = videoSurface->lockFrame();//lock the memory so renderer 
doesn't access this memory
gsize size = gst_buffer_get_size(buffer);
gst_buffer_extract(buffer, 0, data, size);//extract directly to my memory
videoSurface->unlockFrame();//unlock memory and signal to redraw

in draw method I call where data is pointer which returned by lockFrame()

for(uint i=0;i<GST_VIDEO_INFO_N_PLANES(&videoInfo);i++)
             {
                 glActiveTexture(GL_TEXTURE0+i);
                 glBindTexture(GL_TEXTURE_2D, textures[i]);
             glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 
GST_VIDEO_INFO_COMP_WIDTH(&videoInfo, i),
GST_VIDEO_INFO_COMP_HEIGHT(&videoInfo, i), GL_RED, GL_UNSIGNED_BYTE, 
data+GST_VIDEO_INFO_PLANE_OFFSET(&videoInfo, i));
}

Dňa 13.02.2014 17:57, Gordon Nickel wrote / napísal(a):
> Thanks Dusan, that's very helpful.
>
> How are you getting the video data from Gstreamer to OpenGL? I suspect that might be where the slowdown might be happening in my code, since the copy is on the main thread.
>
> Here's the implementation I'm using: https://gist.github.com/gordeaoux/8978998
>
> Which is based on this cinder wrapper: https://github.com/cadet/_2RealGStreamerWrapper/blob/master/src/_2RealGStreamerWrapper.cpp
>
> Gordon Nickel
>
> -----Original Message-----
> From: gstreamer-devel-bounces at lists.freedesktop.org [mailto:gstreamer-devel-bounces at lists.freedesktop.org] On Behalf Of Dušan Poizl
> Sent: Thursday, February 13, 2014 8:26 AM
> To: Discussion of the development of and with GStreamer
> Subject: Re: Poor HD performance with appsink
>
> I used QtGstreamer but because it is not for gstreamer-1.0 I must rewrite my SW to use gstreamer directly. But you can get general idea from this code http://cgit.freedesktop.org/gstreamer/qt-gstreamer/tree/elements/gstqtvideosink/painters/openglsurfacepainter.cpp
> which is where I started when I wrote my my own implementation.
>
> Dňa 13.02.2014 16:45, Gordon Nickel wrote / napísal(a):
>> Thanks for the responses. I'm using callbacks, not signals already.
>>
>> Dusan, how are you displaying content in OpenGL? Is there an implementation you would be willing to share?
>>
>> Gordon Nickel
>>
>> downstream.com
>>
>>> On Feb 13, 2014, at 7:39 AM, "Dušan Poizl" <poizl at maindata.sk> wrote:
>>>
>>> I am using appsink for playing video with OpenGL and have no issue. I can easily play two FullHD video fine while capturing output and pushing to appsrc to encode.
>>>
>>> Dňa 13.02.2014 16:14, Aleix Conchillo Flaqué wrote / napísal(a):
>>>> On Wed, Feb 12, 2014 at 5:40 PM, Gordon Nickel
>>>> <gordon.nickel at downstream.com> wrote:
>>>>> Hi,
>>>>>
>>>>> I have an implementation of Gstreamer 1.0 for Windows using appsink
>>>>> to play videos back. Generally, performance is pretty good, and
>>>>> quality is excellent. However, with some specific videos (all
>>>>> 1080p, 720p seems to be fine), the video track can get very far
>>>>> behind the audio, at which time the audio starts to stutter and
>>>>> eventually cut out. I don't believe it's an issue with the app
>>>>> itself (which generally gets 90-200 fps with videos running), but
>>>>> my implementation of Gstreamer. The same videos play without issue using playbin in gst-launch.
>>>>>
>>>>> I've tried setting drop frames ( gst_app_sink_set_drop to true )
>>>>> and max lateness to various values from 1 second to 0.01 seconds.
>>>>> In doing so, the audio won't cut out, but the video will stutter to
>>>>> an unnacceptable degree (approx 1 frame per second). I've also
>>>>> tried different bit rates, but have generally stuck with h.264 for
>>>>> codecs. Mpeg 2 had similar performance issues.
>>>>>
>>>>> Playing the video through the app but with it's own directdrawsink
>>>>> window played back smoothly.
>>>>>
>>>>> I can provide the problem videos if that would be helpful.
>>>>>
>>>>> Gstreamer builds 1.0.10, 1.2.0 and 1.2.3 have all been tried with
>>>>> the same results.
>>>>>
>>>>> Any ideas to improve performance?
>>>> You might already be doing it, but do you use signals instead of
>>>> callbacks? If so, use callbacks instead. Much faster.
>>>>
>>>> Aleix
>>>> _______________________________________________
>>>> gstreamer-devel mailing list
>>>> gstreamer-devel at lists.freedesktop.org
>>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>> gstreamer-devel at lists.freedesktop.org
>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel