What's the meaning of video/x-raw(memory:GLMemory) without texture-target?
Milian Wolff
milian.wolff at kdab.com
Mon Nov 25 09:30:33 UTC 2019
On Friday, November 22, 2019 5:12:57 AM CET Matthew Waters wrote:
> Hi,
Hey Matt!
> By default a 'video/x-raw(memory:GLMemory)' caps without the
> texture-target field is equivalent to texture-target=2D as that was the
> default before we differentiated the different OpenGL texture targets.
> Now elements should be able to negotiate the differences between them
> correctly however there may be bugs.
OK, thanks - good to know!
> The GLMemory output from omxh264dec is highly dependant on the platform
> you are running on and may or may not work correctly. I know that it
> used to work on the RPi and have not tested any other omx platform myself.
What's interesting is that this seems to work fine when I shove the data
directly into the glimagesink, i.e.:
```
# GST_DEBUG=3 gst-launch-1.0 rpicamsrc ! "video/x-h264,width=640,height=480" !
h264parse ! omxh264dec ! "video/x-raw(memory:GLMemory)" ! queue !
glimagesinkSetting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
0:00:00.488648854 31116 0x167e7b0 FIXME default gstutils.c:
3981:gst_pad_create_stream_id_internal:<rpicamsrc0:src> Creating random
stream-id, consider implementing a deterministic way of creating a stream-id
Got context from element 'sink': gst.gl.GLDisplay=context,
gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayEGL\)\ gldisplayegl0";
Setting pipeline to PLAYING ...
New clock: GstSystemClock
0:00:00.894124323 31116 0x167e7b0 ERROR omxvideodec
gstomxvideodec.c:2079:gst_omx_video_dec_negotiate:<omxh264dec-omxh264dec0>
Empty caps
0:00:00.985837604 31116 0x167e230 FIXME glslstage
gstglslstage.c:518:_compile_shader:<glslstage0> vertex shader info
log:Compiled
0:00:00.990119375 31116 0x167e230 FIXME glslstage
gstglslstage.c:518:_compile_shader:<glslstage1> fragment shader info
log:Compiled
0:00:01.013341718 31116 0x167e230 FIXME glslstage
gstglslstage.c:518:_compile_shader:<glslstage2> vertex shader info
log:Compiled
0:00:01.014671666 31116 0x167e230 FIXME glslstage
gstglslstage.c:518:_compile_shader:<glslstage3> fragment shader info
log:Compiled
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:06.801416612
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
```
I don't know what the ERROR is trying to tell me, but this pipeline works a
charm.
> The first step to debugging this would be to enable GStreamer debug
> logging to get an idea of what may be wrong.
But the following happens when I add the gleffects_twirl filter (either before
or after the queue, doesn't matter):
```
# GST_DEBUG=3 gst-launch-1.0 rpicamsrc ! "video/x-h264,width=640,height=480" !
h264parse ! omxh264dec ! "video/x-raw(memory:GLMemory)" ! gleffects_twirl !
queue ! glimagesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
0:00:00.506736041 31143 0x1fed490 FIXME default gstutils.c:
3981:gst_pad_create_stream_id_internal:<rpicamsrc0:src> Creating random
stream-id, consider implementing a deterministic way of creating a stream-id
Got context from element 'sink': gst.gl.GLDisplay=context,
gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayEGL\)\ gldisplayegl0";
Setting pipeline to PLAYING ...
New clock: GstSystemClock
0:00:00.882512708 31143 0x1fed490 ERROR omxvideodec
gstomxvideodec.c:2079:gst_omx_video_dec_negotiate:<omxh264dec-omxh264dec0>
Empty caps
0:00:00.968698020 31143 0x6fd03290 WARN videodecoder
gstvideodecoder.c:3783:gst_video_decoder_negotiate_pool:<omxh264dec-
omxh264dec0> Subclass failed to decide allocation
0:00:00.969139166 31143 0x6fd03290 WARN GST_PADS gstpad.c:
4231:gst_pad_peer_query:<omxh264dec-omxh264dec0:src> could not send sticky
events
0:00:00.969251770 31143 0x6fd03290 WARN videodecoder
gstvideodecoder.c:3783:gst_video_decoder_negotiate_pool:<omxh264dec-
omxh264dec0> Subclass failed to decide allocation
0:00:00.972893854 31143 0x6fd03290 WARN GST_PADS gstpad.c:
4231:gst_pad_peer_query:<omxh264dec-omxh264dec0:src> could not send sticky
events
0:00:00.988252083 31143 0x6fd03290 WARN omxvideodec
gstomxvideodec.c:1927:gst_omx_video_dec_loop:<omxh264dec-omxh264dec0> error:
Internal data stream error.
0:00:00.988374999 31143 0x6fd03290 WARN omxvideodec
gstomxvideodec.c:1927:gst_omx_video_dec_loop:<omxh264dec-omxh264dec0> error:
stream stopped, reason not-negotiated
0:00:00.988662604 31143 0x1fed490 WARN basesrc gstbasesrc.c:
3072:gst_base_src_loop:<rpicamsrc0> error: Internal data stream error.
0:00:00.988741718 31143 0x1fed490 WARN basesrc gstbasesrc.c:
3072:gst_base_src_loop:<rpicamsrc0> error: streaming stopped, reason not-
negotiated (-4)
ERROR: from element /GstPipeline:pipeline0/GstOMXH264Dec-
omxh264dec:omxh264dec-omxh264dec0: Internal data stream error.
Additional debug info:
../../gst-omx-1.16.1/omx/gstomxvideodec.c(1927): gst_omx_video_dec_loop (): /
GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0:
stream stopped, reason not-negotiated
Execution ended after 0:00:00.478420208
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
```
Do you have an idea of what's going on here?
Thanks
> On 22/11/19 3:10 am, Milian Wolff wrote:
> > Hey all,
> >
> > on a Raspberry Pi 3b+ gst-inspect-1.0 shows the following src for
> > omxh264dec:
> >
> > ```
> >
> > video/x-raw(memory:GLMemory)
> >
> > format: RGBA
> >
> > width: [ 1, 2147483647 ]
> >
> > height: [ 1, 2147483647 ]
> >
> > framerate: [ 0/1, 2147483647/1 ]
> >
> > ```
> >
> > Note that this is missing a texture-target specification. What kind of
> > memory is this, and how can I leverage it further down? I would like to
> > efficiently change video frames using an OpenGL filter. If we take
> > gleffects-twirl as an example, it expects the following as sink:
> >
> >
> > ```
> >
> > video/x-raw(memory:GLMemory):
> > format: RGBA
> >
> > width: [ 1, 2147483647 ]
> >
> > height: [ 1, 2147483647 ]
> >
> > framerate: [ 0/1, 2147483647/1 ]
> >
> > texture-target: 2D
> >
> > ```
> >
> > How can I connect the two together? If I try, to add the filter directly
> > after the decoder in a pipeline, I get:
> >
> > ```
> > ERROR: from element /GstPipeline:pipeline0/GstOMXH264Dec-
> > omxh264dec:omxh264dec-omxh264dec0: Internal data stream error.
> > Additional debug info:
> > ../../gst-omx-1.16.1/omx/gstomxvideodec.c(1927): gst_omx_video_dec_loop
> > (): /
> > GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0:
> > stream stopped, reason not-negotiated
> > ```
> >
> > Can someone sched some light on this? How can one post-process the
> > GLMemory in a gl video filter? Is the missing texture-target a bug in
> > omxh264dec, or can I somehow make the raw GLMemory a texture? Or does
> > this mean the frame lives in some other form of non-texture memory that
> > can be displayed, but not otherwise accessed by OpenGL for video filter
> > purposes?
> >
> > Thanks
> >
> > PS: In case you are wondering what we are trying to achieve on a higher
> > level:
> >
> > rpicamsrc -> h264 -> omxh264dec -> glmemory -> custom gl filter to add a
> > non- static overlay to every frame -> tee to display sink and then also a
> > h264- encoded avi record sink
> >
> > _______________________________________________
> > gstreamer-devel mailing list
> > gstreamer-devel at lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
--
Milian Wolff | milian.wolff at kdab.com | Senior Software Engineer
KDAB (Deutschland) GmbH, a KDAB Group company
Tel: +49-30-521325470
KDAB - The Qt, C++ and OpenGL Experts
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3826 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20191125/8a0a2c06/attachment.bin>
More information about the gstreamer-devel
mailing list