encoding pipeline from v4l2 usb camera with v4l2video8convert and v4l2h264enc

Nicolas Dufresne nicolas at ndufresne.ca
Sun Jan 19 17:57:39 UTC 2020


Le dimanche 19 janvier 2020 à 15:12 +0200, Constantine Elster a écrit :
> Thank you very much Milian! Looks better with h264parse. Still getting an error, though now different one.
> 
> I added h264parse to the pipeline:
> gst-launch-1.0 -vvv v4l2src device="/dev/video9" num-buffers=200 ! "video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1, colorimetry=bt709" ! v4l2video8convert output-io-mode=dmabuf-import ! v4l2h264enc output-io-mode=dmabuf-import ! h264parse ! mp4mux ! filesink location=aha.mp4
> 
> Output:
> Setting pipeline to PAUSED ...
> Pipeline is live and does not need PREROLL ...
> Setting pipeline to PLAYING ...
> New clock: GstSystemClock
> /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
> /GstPipeline:pipeline0/v4l2video8convert:v4l2video8convert0.GstPad:src: caps = video/x-raw, framerate=(fraction)60/1, interlace-mode=(string)progressive, format=(string)I420, width=(int)640, height=(int)480, colorimetry=(string)bt709
> /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)baseline, level=(string)4, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> /GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)baseline, level=(string)4, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> Redistribute latency...
> /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0.GstPad:sink: caps = video/x-raw, framerate=(fraction)60/1, interlace-mode=(string)progressive, format=(string)I420, width=(int)640, height=(int)480, colorimetry=(string)bt709
> /GstPipeline:pipeline0/v4l2video8convert:v4l2video8convert0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
> ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
> Additional debug info:
> gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
> streaming stopped, reason error (-5)

Looks like the real error was not propagated (let me know which version you are
using, Ubuntu tends to be far behind, or only cherry-picking fixes rather then
tracking stable branches).

To find our more, set GST_DEBUG="v4l2*:7" env. That will give you a lot more
details. By experience, I suspect that your output-io-mode=dmabuf-import is
faulty on the v4l2convert element (on newer GStreamer, we now have a fix name
for converter too). 

IMX.6 platform (and IMX.8 too) didn't include an IOMMU in their platform. On the
other side, USB cameras (like UVC) produces scattered memory (virtual memory).
That makes importation of dmabuf from UVC to v4l2convert impossible in this
case. You may want to try the other way around, to set io-mode=dmabuf-import on
the v4l2src element. If that does not work, you'll have no choice but to let the
pipelein makes a copy (default setting). Normally, I suggest to first try
without touching these advance settings, to make it work, and then tweak to gain
more performance.

> Execution ended after 0:00:00.436501205
> Setting pipeline to PAUSED ...
> Setting pipeline to READY ...
> Setting pipeline to NULL ...
> Freeing pipeline ...
> 
> Any hint on debugging would be appreciated!
> Thank you,
>   -- Constantine.
> 
> 
> 
> 
> 
> On Sun, Jan 19, 2020 at 2:13 PM Milian Wolff <milian.wolff at kdab.com> wrote:
> > On Sonntag, 19. Januar 2020 11:06:55 CET Constantine Elster wrote:
> > > Hi devs,
> > > 
> > > I'm trying to construct a pipeline that captures frames from a USB camera
> > > (YUV) and encodes them with HW encoder and saves into a file. My setup is
> > > iMX6 board running Ubuntu 18.04 on 4.20 mainline kernel.
> > > 
> > > When I try a sw encoder, it works okay albeit I get very high 100% CPU
> > > usage. The working pipeline based on software plugins:
> > > gst-launch-1.0 -v v4l2src device="/dev/video2" num-buffers=200 !
> > > "video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480,
> > > pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)45/1,
> > > colorimetry=bt709" ! videoconvert ! x264enc ! mp4mux ! filesink
> > > location=aha.mp4
> > > 
> > > My attempt to replace sw based plugins by HW based with efficient memory
> > > management:
> > > gst-launch-1.0 -v v4l2src device="/dev/video9" num-buffers=200 !
> > > "video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480,
> > > pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)45/1,
> > > colorimetry=bt709" ! *v4l2video8convert* *output-io-mode=dmabuf-import* !
> > > *v4l2h264enc* *output-io-mode=dmabuf-import* ! mp4mux ! filesink
> > > location=aha.mp4
> > > 
> > > I get the following error: "WARNING: erroneous pipeline: could not link
> > > v4l2h264enc0 to mp4mux0"
> > > 
> > > Would appreciate any ideas how to understand what's wrong, how to debug and
> > > make it work.
> > 
> > To debug, I suggest you compare the SRC of `gst-inspect-1.0 v4l2h264enc` with 
> > the SINK of `gst-inspect-1.0 mp4mux`. My guess is that you may be able to fix 
> > the issue by adding a `h264parse` element in the middle to fix the alignment 
> > since `mp4mux` requires `au` alignment, whereas the encoder may output `nal` 
> > frames?
> > 
> > Good luck
> > 
> > -- 
> > Milian Wolff | milian.wolff at kdab.com | Senior Software Engineer
> > KDAB (Deutschland) GmbH, a KDAB Group company
> > Tel: +49-30-521325470
> > KDAB - The Qt, C++ and OpenGL Experts
> 
> 
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel



More information about the gstreamer-devel mailing list