Errors with pipelines and CODA on i.MX6 with mainline kernel

[Christopher Held (sks Kinkel)]

Hi,

I have some problems using GStreamer pipelines and using CODA driver (for h264 en-/decoding).
First of all, my setup:
I'm using a Ka-Ro TX6S 8035 board which uses a i.MX6 Solo CPU.
On that board, I am running Yocto Linux (version 2.4 Rocko) with mainline kernel 4.14.24.
I am not using the meta-freescale layer currently, but I include the i.MX6-specific firmware to use CODA.
I created a relatively basic image with no X11 or Wayland support, just GStreamer, some command-line tools and some QT recipes (intended for eglfs support).
Using Yocto, I install the following GStreamer packages (coming from openembedded-core layer with version 1.12.2):
gstreamer1.0
gstreamer1.0-meta-base
gstreamer1.0-plugins-base
gstreamer1.0-plugins-good
gstreamer1.0-plugins-bad
gstreamer1.0-plugins-ugly
gstreamer1.0-vaapi

Also, CODA seems to be properly loaded, according to this part during bootup:
coda 2040000.vpu: Direct firmware load for vpu_fw_imx6d.bin failed with error -2
coda 2040000.vpu: Falling back to user helper
coda 2040000.vpu: Using fallback firmware vpu/vpu_fw_imx6d.bin
coda 2040000.vpu: Firmware code revision: 46072
coda 2040000.vpu: Initialized CODA960.
coda 2040000.vpu: Firmware version: 3.1.1
coda 2040000.vpu: codec registered as /dev/video[0-1]


Now to my problem(s):
1. No kmssink, although gstreamer1.0-plugins-bad installed.
Online, I've seen multiple GStreamer pipelines suggested and said to be working on similar setups using the pipeline:
gst-launch-1.0 filesrc location=/root/trailer_1080p_h264_mp3.avi ! \
avidemux ! h264parse ! v4l2video1dec capture-io-mode=dmabuf ! \
kmssink

But when inspecting my GStreamer plugins, I don't have kmssink. Following is a grep for sinks:
root at imx6dl-tx6-emmc:~# gst-inspect-1.0 | grep sink
video4linux2:  v4l2sink: Video (video4linux2) Sink
debug:  testsink: Test plugin
xvimagesink:  xvimagesink: Video sink
curl:  curlhttpsink: Curl http sink
curl:  curlfilesink: Curl file sink
curl:  curlftpsink: Curl ftp sink
curl:  curlsmtpsink: Curl smtp sink
decklink:  decklinkaudiosink: Decklink Audio Sink
decklink:  decklinkvideosink: Decklink Video Sink
opengl:  glimagesink: GL Sink Bin
opengl:  glimagesinkelement: OpenGL video sink
opengl:  glsinkbin: GL Sink Bin
debugutilsbad:  checksumsink: Checksum sink
debugutilsbad:  fpsdisplaysink: Measure and show framerate on videosink
alsa:  alsasink: Audio sink (ALSA)
autodetect:  autovideosink: Auto video sink
autodetect:  autoaudiosink: Auto audio sink
gio:  giosink: GIO sink
gio:  giostreamsink: GIO stream sink
ximagesink:  ximagesink: Video sink
tcp:  tcpclientsink: TCP client sink
tcp:  tcpserversink: TCP server sink
tcp:  multifdsink: Multi filedescriptor sink
tcp:  multisocketsink: Multi socket sink
fbdevsink:  fbdevsink: fbdev video sink
gdkpixbuf:  gdkpixbufsink: GdkPixbuf sink
app:  appsink: AppSink
soup:  souphttpclientsink: HTTP client sink
waylandsink:  waylandsink: wayland video sink
bluez:  a2dpsink: Bluetooth A2DP sink
bluez:  avdtpsink: Bluetooth AVDTP sink
hls:  hlssink: HTTP Live Streaming sink
playback:  playsink: Player Sink
ossaudio:  osssink: Audio Sink (OSS)
udp:  udpsink: UDP packet sender
udp:  multiudpsink: UDP packet sender
udp:  dynudpsink: UDP packet sender
shm:  shmsink: Shared Memory Sink
vulkan:  vulkansink: Vulkan video sink
coreelements:  fakesink: Fake Sink
coreelements:  fdsink: Filedescriptor Sink
coreelements:  filesink: File Sink
inter:  interaudiosink: Internal audio sink
inter:  intersubsink: Internal subtitle sink
inter:  intervideosink: Internal video sink
multifile:  multifilesink: Multi-File Sink
multifile:  splitmuxsink: Split Muxing Bin

Now, as far as I know, kmssink is part of gstreamer1.0-plugins-bad, which I install. Other parts of the bad plugins, like h264parse, do exist. Are there maybe other requirements for Yocto-based images, that I don't know of, to make kmssink available?

2. Delayed linking error when using avidemux.
When trying to manually create a pipeline to play a test .avi file, I get the following error in all variations of pipelines I tested:
root at imx6dl-tx6-emmc:~# gst-launch-1.0 filesrc location=/usr/share/movies/big_buck_bunny_480p_surround-fix.avi ! avidemux ! h264parse ! v4l2video1dec ! autovideosink
Setting pipeline to PAUSED ...
error: XDG_RUNTIME_DIR not set in the environment.
error: XDG_RUNTIME_DIR not set in the environment.
Pipeline is PREROLLING ...
WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0: Could not initialise Xv output
Additional debug info:
../../../gst-plugins-base-1.12.2/sys/xvimage/xvimagesink.c(1759): gst_xv_image_sink_open (): /GstXvImageSink:autovideosink0-actual-sink-xvimage:
Could not open display (null)
WARNING: from element /GstPipeline:pipeline0/GstAviDemux:avidemux0: Delayed linking failed.
Additional debug info:
../../../gstreamer-1.12.2/gst/parse/grammar.y(510): gst_parse_no_more_pads (): /GstPipeline:pipeline0/GstAviDemux:avidemux0:
failed delayed linking some pad of GstAviDemux named avidemux0 to some pad of GstH264Parse named h264parse0
ERROR: from element /GstPipeline:pipeline0/GstAviDemux:avidemux0: Internal data stream error.
Additional debug info:
../../../gst-plugins-good-1.12.2/gst/avi/gstavidemux.c(5770): gst_avi_demux_loop (): /GstPipeline:pipeline0/GstAviDemux:avidemux0:
streaming stopped, reason not-linked (-1)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...

Is avidemux not the right choice when trying to play a .avi file? What going wrong here? Since the same error occurs with queue between avidemux and h264parse, I assume h264parse is not the problem here?

3. Using gst-play-1.0, CODA throws an error.
root at imx6dl-tx6-emmc:~# gst-play-1.0 /usr/share/movies/big_buck_bunny_480p_surround-fix.avi
Press 'k' to see a list of keyboard shortcuts.
Now playing /usr/share/movies/big_buck_bunny_480p_surround-fix.avi
Prerolling...
error: XDG_RUNTIME_DIR not set in the environment.
error: XDG_RUNTIME_DIR not set in the environment.
WARNING Could not initialise Wayland output
WARNING debug information: ../../../gst-plugins-bad-1.12.2/ext/wayland/gstwaylandsink.c(294): gst_wayland_sink_find_display (): /GstWaylandSink:waylandsink0:
Failed to create GstWlDisplay: 'Failed to connect to the wayland display '(default)''
No accelerated IMDCT transform found
error: XDG_RUNTIME_DIR not set in the environment.
error: XDG_RUNTIME_DIR not set in the environment.
0:00:00.0 / 0:09:56.4
0:00:00.0 / 0:09:56.4
coda 2040000.vpu: CODA_COMMAND_SEQ_INIT failed, error code = 8192

Searching online for this error didn't wield results (or I need to step up my search-game?) and I also couldn't find others with similar error messages.

If this is not the right place for CODA error related questions, I would be grateful if you could point me to where I can get support for those. Thank you.

Best regards,
Christopher Held