Appsink Vs. Last-Sample

[Brendan Lockhart]

>> I am using Gstreamer to render live video with Unity3D. Currently, I
append
>> a fakesink to my pipeline, and at every frame of the Unity application
the
>> fakesink's last-sample is queried and used to update a texture.
>>
>> This works, but I feel that the performance of the plugin is a bit
lacking.
>> I'm considering rewriting the plugin to instead use an appsink.
>>
>> My question is: does querying appsink samples have any different
performance
>> characteristics relative to querying a fakesink's last-sample?
>>
>Are you connecting to the last-sample signal and running a callback?
>If so, there is a noticeable difference between using a gobject signal
>for running a callback vs using a function call in libgstapp. You can
>try using gst_app_sink_pull_sample() and friends.

Thanks for the reply!

Every frame of the application, the plugin get's the fakesink's
"last-sample" property. From the returned GstSample it gets the buffer and
copies that to a target texture, so it's not using any callbacks.

>However, I think you should do some performance measurements first. I
>suspect that since you're doing GL rendering, most of your overhead is
>in copying video buffers around and the function call vs. signal
>callback overhead is minor in comparison. In that case, you should be
>using glimagesink to write directly to a GL texture. Here's some
>examples that do this with SDL2:
>https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree
/tests/examples/gl/sdl

You're probably right that the major bottleneck is most likely copying the
data from the CPU to the GPU. I looked into it, and I think I can have
gstamcvideodec decode directly into GL memory.
Unfortunately when I try to do this gstamcvideodec logs:
"ERR amcvideodec
gstamcvideodec.c:1977:gst_amc_video_dec_set_format:<amcvideodec-omxexynosavcdec0>
Codec only supports GL output but downstream does not"

This is despite that fact that, before that error, gstamcvideodec logs:

"gstamcvideodec.c:1904:gst_amc_video_dec_set_format:<amcvideodec-omxexynosavcdec0>
Available downstream caps: video/x-raw(memory:GLMemory),
format=(string)RGBA, texture-target=(string)external-oes"

Which I *believe *should intersect properly with amcvideodec's required GL
output sink caps.

Could this be a bug, or am I doing something wrong here?
This is with Gstreamer 1.10.4, and I'm setting playbin's videosink to
"video/x-raw(memory:GLMemory),texture-target=2D,format=RGBA ! fakesink
sync=1 qos=1 name=sink"

On Thu, Mar 16, 2017 at 9:12 AM, Nirbheek Chauhan <
nirbheek.chauhan at gmail.com> wrote:

> On Thu, Mar 16, 2017 at 8:49 AM, Brendan Lockhart <somedude114 at gmail.com>
> wrote:
> > I am using Gstreamer to render live video with Unity3D. Currently, I
> append
> > a fakesink to my pipeline, and at every frame of the Unity application
> the
> > fakesink's last-sample is queried and used to update a texture.
> >
> > This works, but I feel that the performance of the plugin is a bit
> lacking.
> > I'm considering rewriting the plugin to instead use an appsink.
> >
> > My question is: does querying appsink samples have any different
> performance
> > characteristics relative to querying a fakesink's last-sample?
> >
>
> Are you connecting to the last-sample signal and running a callback?
> If so, there is a noticeable difference between using a gobject signal
> for running a callback vs using a function call in libgstapp. You can
> try using gst_app_sink_pull_sample() and friends.
>
> However, I think you should do some performance measurements first. I
> suspect that since you're doing GL rendering, most of your overhead is
> in copying video buffers around and the function call vs. signal
> callback overhead is minor in comparison. In that case, you should be
> using glimagesink to write directly to a GL texture. Here's some
> examples that do this with SDL2:
> https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/
> tests/examples/gl/sdl
>
> Cheers,
> Nirbheek
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20170318/1fc4dc2e/attachment.html>