Poor HD performance with appsink

[Dušan Poizl]

I display I420 and RGB frames. I upload YUV planes to three GL_RED 
textures with glTexSubImage. Then draw single quad with custom shader 
which just sample from this three textures and convert it to RGB by 
multiply YUV by color conversion matrix. You just do M*vec4(Y, U, V, 
1.0); in GLSL shader where M is one of this

     case GST_VIDEO_COLOR_MATRIX_BT709:
         colorMatrix = QMatrix4x4(1.164383561643836, 0.0, 
1.792741071428571, -0.972945075016308,
                       1.164383561643836, -0.213248614273730, 
-0.532909328559444, 0.301482665475862,
                       1.164383561643836, 2.112401785714286, 0.0, 
-1.133402217873451,
                       0.0, 0.0, 0.0, 1.0);
         break;
     case GST_VIDEO_COLOR_MATRIX_BT601:
         colorMatrix = QMatrix4x4(1.16438356164384, 0.0, 
1.59602678571429,  -0.874202217873451,
                        1.16438356164384, -0.391762290094914, 
-0.812967647237771, 0.531667823499146,
                        1.16438356164384, 2.01723214285714, 0.0, 
-1.08563078930202,
                        0.0, 0.0, 0.0, 1.0);

they are derived from matrix that you can find 
http://en.wikipedia.org/wiki/YUV This matrix incorporate scaling from 
16-235 range to 0.0-1.0

Dňa 13.02.2014 16:45, Gordon Nickel wrote / napísal(a):
> Thanks for the responses. I'm using callbacks, not signals already.
>
> Dusan, how are you displaying content in OpenGL? Is there an implementation you would be willing to share?
>
> Gordon Nickel
>
> downstream.com
>
>> On Feb 13, 2014, at 7:39 AM, "Dušan Poizl" <poizl at maindata.sk> wrote:
>>
>> I am using appsink for playing video with OpenGL and have no issue. I can easily play two FullHD video fine while capturing output and pushing to appsrc to encode.
>>
>> Dňa 13.02.2014 16:14, Aleix Conchillo Flaqué wrote / napísal(a):
>>> On Wed, Feb 12, 2014 at 5:40 PM, Gordon Nickel
>>> <gordon.nickel at downstream.com> wrote:
>>>> Hi,
>>>>
>>>> I have an implementation of Gstreamer 1.0 for Windows using appsink to play
>>>> videos back. Generally, performance is pretty good, and quality is
>>>> excellent. However, with some specific videos (all 1080p, 720p seems to be
>>>> fine), the video track can get very far behind the audio, at which time the
>>>> audio starts to stutter and eventually cut out. I don't believe it's an
>>>> issue with the app itself (which generally gets 90-200 fps with videos
>>>> running), but my implementation of Gstreamer. The same videos play without
>>>> issue using playbin in gst-launch.
>>>>
>>>> I've tried setting drop frames ( gst_app_sink_set_drop to true ) and max
>>>> lateness to various values from 1 second to 0.01 seconds. In doing so, the
>>>> audio won't cut out, but the video will stutter to an unnacceptable degree
>>>> (approx 1 frame per second). I've also tried different bit rates, but have
>>>> generally stuck with h.264 for codecs. Mpeg 2 had similar performance
>>>> issues.
>>>>
>>>> Playing the video through the app but with it's own directdrawsink window
>>>> played back smoothly.
>>>>
>>>> I can provide the problem videos if that would be helpful.
>>>>
>>>> Gstreamer builds 1.0.10, 1.2.0 and 1.2.3 have all been tried with the same
>>>> results.
>>>>
>>>> Any ideas to improve performance?
>>> You might already be doing it, but do you use signals instead of
>>> callbacks? If so, use callbacks instead. Much faster.
>>>
>>> Aleix
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>> gstreamer-devel at lists.freedesktop.org
>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel