What's the meaning of video/x-raw(memory:GLMemory) without texture-target?
Matthew Waters
ystreet00 at gmail.com
Mon Nov 25 10:20:00 UTC 2019
Try adding a glupload ! glcolorconvert before the gleffects_twirl
element. glimagesink contains these automatically already.
On 25/11/19 8:30 pm, Milian Wolff wrote:
> On Friday, November 22, 2019 5:12:57 AM CET Matthew Waters wrote:
>> Hi,
> Hey Matt!
>
>> By default a 'video/x-raw(memory:GLMemory)' caps without the
>> texture-target field is equivalent to texture-target=2D as that was the
>> default before we differentiated the different OpenGL texture targets.
>> Now elements should be able to negotiate the differences between them
>> correctly however there may be bugs.
> OK, thanks - good to know!
>
>> The GLMemory output from omxh264dec is highly dependant on the platform
>> you are running on and may or may not work correctly. I know that it
>> used to work on the RPi and have not tested any other omx platform myself.
> What's interesting is that this seems to work fine when I shove the data
> directly into the glimagesink, i.e.:
>
> ```
> # GST_DEBUG=3 gst-launch-1.0 rpicamsrc ! "video/x-h264,width=640,height=480" !
> h264parse ! omxh264dec ! "video/x-raw(memory:GLMemory)" ! queue !
> glimagesinkSetting pipeline to PAUSED ...
> Pipeline is live and does not need PREROLL ...
> 0:00:00.488648854 31116 0x167e7b0 FIXME default gstutils.c:
> 3981:gst_pad_create_stream_id_internal:<rpicamsrc0:src> Creating random
> stream-id, consider implementing a deterministic way of creating a stream-id
> Got context from element 'sink': gst.gl.GLDisplay=context,
> gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayEGL\)\ gldisplayegl0";
> Setting pipeline to PLAYING ...
> New clock: GstSystemClock
> 0:00:00.894124323 31116 0x167e7b0 ERROR omxvideodec
> gstomxvideodec.c:2079:gst_omx_video_dec_negotiate:<omxh264dec-omxh264dec0>
> Empty caps
> 0:00:00.985837604 31116 0x167e230 FIXME glslstage
> gstglslstage.c:518:_compile_shader:<glslstage0> vertex shader info
> log:Compiled
> 0:00:00.990119375 31116 0x167e230 FIXME glslstage
> gstglslstage.c:518:_compile_shader:<glslstage1> fragment shader info
> log:Compiled
> 0:00:01.013341718 31116 0x167e230 FIXME glslstage
> gstglslstage.c:518:_compile_shader:<glslstage2> vertex shader info
> log:Compiled
> 0:00:01.014671666 31116 0x167e230 FIXME glslstage
> gstglslstage.c:518:_compile_shader:<glslstage3> fragment shader info
> log:Compiled
> ^Chandling interrupt.
> Interrupt: Stopping pipeline ...
> Execution ended after 0:00:06.801416612
> Setting pipeline to PAUSED ...
> Setting pipeline to READY ...
> Setting pipeline to NULL ...
> Freeing pipeline ...
> ```
>
> I don't know what the ERROR is trying to tell me, but this pipeline works a
> charm.
>
>> The first step to debugging this would be to enable GStreamer debug
>> logging to get an idea of what may be wrong.
> But the following happens when I add the gleffects_twirl filter (either before
> or after the queue, doesn't matter):
>
> ```
> # GST_DEBUG=3 gst-launch-1.0 rpicamsrc ! "video/x-h264,width=640,height=480" !
> h264parse ! omxh264dec ! "video/x-raw(memory:GLMemory)" ! gleffects_twirl !
> queue ! glimagesink
> Setting pipeline to PAUSED ...
> Pipeline is live and does not need PREROLL ...
> 0:00:00.506736041 31143 0x1fed490 FIXME default gstutils.c:
> 3981:gst_pad_create_stream_id_internal:<rpicamsrc0:src> Creating random
> stream-id, consider implementing a deterministic way of creating a stream-id
> Got context from element 'sink': gst.gl.GLDisplay=context,
> gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayEGL\)\ gldisplayegl0";
> Setting pipeline to PLAYING ...
> New clock: GstSystemClock
> 0:00:00.882512708 31143 0x1fed490 ERROR omxvideodec
> gstomxvideodec.c:2079:gst_omx_video_dec_negotiate:<omxh264dec-omxh264dec0>
> Empty caps
> 0:00:00.968698020 31143 0x6fd03290 WARN videodecoder
> gstvideodecoder.c:3783:gst_video_decoder_negotiate_pool:<omxh264dec-
> omxh264dec0> Subclass failed to decide allocation
> 0:00:00.969139166 31143 0x6fd03290 WARN GST_PADS gstpad.c:
> 4231:gst_pad_peer_query:<omxh264dec-omxh264dec0:src> could not send sticky
> events
> 0:00:00.969251770 31143 0x6fd03290 WARN videodecoder
> gstvideodecoder.c:3783:gst_video_decoder_negotiate_pool:<omxh264dec-
> omxh264dec0> Subclass failed to decide allocation
> 0:00:00.972893854 31143 0x6fd03290 WARN GST_PADS gstpad.c:
> 4231:gst_pad_peer_query:<omxh264dec-omxh264dec0:src> could not send sticky
> events
> 0:00:00.988252083 31143 0x6fd03290 WARN omxvideodec
> gstomxvideodec.c:1927:gst_omx_video_dec_loop:<omxh264dec-omxh264dec0> error:
> Internal data stream error.
> 0:00:00.988374999 31143 0x6fd03290 WARN omxvideodec
> gstomxvideodec.c:1927:gst_omx_video_dec_loop:<omxh264dec-omxh264dec0> error:
> stream stopped, reason not-negotiated
> 0:00:00.988662604 31143 0x1fed490 WARN basesrc gstbasesrc.c:
> 3072:gst_base_src_loop:<rpicamsrc0> error: Internal data stream error.
> 0:00:00.988741718 31143 0x1fed490 WARN basesrc gstbasesrc.c:
> 3072:gst_base_src_loop:<rpicamsrc0> error: streaming stopped, reason not-
> negotiated (-4)
> ERROR: from element /GstPipeline:pipeline0/GstOMXH264Dec-
> omxh264dec:omxh264dec-omxh264dec0: Internal data stream error.
> Additional debug info:
> ../../gst-omx-1.16.1/omx/gstomxvideodec.c(1927): gst_omx_video_dec_loop (): /
> GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0:
> stream stopped, reason not-negotiated
> Execution ended after 0:00:00.478420208
> Setting pipeline to PAUSED ...
> Setting pipeline to READY ...
> Setting pipeline to NULL ...
> Freeing pipeline ...
> ```
>
> Do you have an idea of what's going on here?
>
> Thanks
>
>> On 22/11/19 3:10 am, Milian Wolff wrote:
>>> Hey all,
>>>
>>> on a Raspberry Pi 3b+ gst-inspect-1.0 shows the following src for
>>> omxh264dec:
>>>
>>> ```
>>>
>>> video/x-raw(memory:GLMemory)
>>>
>>> format: RGBA
>>>
>>> width: [ 1, 2147483647 ]
>>>
>>> height: [ 1, 2147483647 ]
>>>
>>> framerate: [ 0/1, 2147483647/1 ]
>>>
>>> ```
>>>
>>> Note that this is missing a texture-target specification. What kind of
>>> memory is this, and how can I leverage it further down? I would like to
>>> efficiently change video frames using an OpenGL filter. If we take
>>> gleffects-twirl as an example, it expects the following as sink:
>>>
>>>
>>> ```
>>>
>>> video/x-raw(memory:GLMemory):
>>> format: RGBA
>>>
>>> width: [ 1, 2147483647 ]
>>>
>>> height: [ 1, 2147483647 ]
>>>
>>> framerate: [ 0/1, 2147483647/1 ]
>>>
>>> texture-target: 2D
>>>
>>> ```
>>>
>>> How can I connect the two together? If I try, to add the filter directly
>>> after the decoder in a pipeline, I get:
>>>
>>> ```
>>> ERROR: from element /GstPipeline:pipeline0/GstOMXH264Dec-
>>> omxh264dec:omxh264dec-omxh264dec0: Internal data stream error.
>>> Additional debug info:
>>> ../../gst-omx-1.16.1/omx/gstomxvideodec.c(1927): gst_omx_video_dec_loop
>>> (): /
>>> GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0:
>>> stream stopped, reason not-negotiated
>>> ```
>>>
>>> Can someone sched some light on this? How can one post-process the
>>> GLMemory in a gl video filter? Is the missing texture-target a bug in
>>> omxh264dec, or can I somehow make the raw GLMemory a texture? Or does
>>> this mean the frame lives in some other form of non-texture memory that
>>> can be displayed, but not otherwise accessed by OpenGL for video filter
>>> purposes?
>>>
>>> Thanks
>>>
>>> PS: In case you are wondering what we are trying to achieve on a higher
>>> level:
>>>
>>> rpicamsrc -> h264 -> omxh264dec -> glmemory -> custom gl filter to add a
>>> non- static overlay to every frame -> tee to display sink and then also a
>>> h264- encoded avi record sink
>>>
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>> gstreamer-devel at lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20191125/561f8b41/attachment.sig>
More information about the gstreamer-devel
mailing list