Raspberry Pi 4 is not working when the camera video size is increased.

[Nicolas Dufresne]

Le vendredi 29 janvier 2021 à 11:28 -0600, iotsystek a écrit :
> I am new to Linux and GStreamer.  I am working on getting 4 - 1 mega pixel
> USB 2.0 cameras working on a RPi4.  Currently I am using just 2 cameras for
> testing.  The cameras are connected via USB 2.0 directly to the Pi (no
> external hub).  The RPi4 has 256Mb GPU memory set and is being powered with
> a precision bench power supply at 5.05 volts and does not complain about low
> voltage.  There is no audio in this project.  My final application will be
> embedded into my diesel truck and 5th wheel.
> 
> I have 2 different hardware test configurations:
> 1       A Raspberry Pi 4 with Buster OS dates Dec 2 2020.
>         Each camera is connected directly to a Pi USB port.
>         GStreamer version is 1.14.4
> 
> 2       Linux Ubuntu 18.04 running in a virtual machine on my PC.
>         1 camera is connected directly to a PC USB 3.0 port.
>         1 camera is connected to a PC via a USB 3.0 hub.
>         GStreamer version is 1.14.5
> 
> The following 2 gst-launch-1.0 commands are being used.  The only difference
> between them are the size of the video 320x240 or 640x480.
> 
> gst-launch-1.0 videomixer name=mix ! videoconvert ! autovideosink sync=false
> \
>  v4l2src device=/dev/video0 ! videorate max-rate=6 ! queue ! videoconvert !
> video/x-raw, format=AYUV, width=320, height=240 ! videobox border-alpha=0
> top=0    left=0    ! mix. \
>  v4l2src device=/dev/video2 ! videorate max-rate=6 ! queue ! videoconvert !
> video/x-raw, format=AYUV, width=320, height=240 ! videobox border-alpha=0
> top=0    left=-320 ! mix.
> 
> gst-launch-1.0 videomixer name=mix ! videoconvert ! ximagesink sync=false \
>  v4l2src device=/dev/video0 ! videorate max-rate=6 ! queue ! videoconvert !
> video/x-raw, format=AYUV, width=640, height=480 ! videobox border-alpha=0
> top=0    left=0    ! mix. \
>  v4l2src device=/dev/video2 ! videorate max-rate=6 ! queue ! videoconvert !
> video/x-raw, format=AYUV, width=640, height=480 ! videobox border-alpha=0
> top=0    left=-640 ! mix.
> 
> In the following write up my definition of working means that the video
> starts up with gst-launch-1.0 and updates as expected when I move the camera
> around. (Real time video)
> 
> Both of the above commands work on the PC VM.
> 
> On the RPi4 the smaller window always works but the larger window only kind
> of works sometimes. With the larger window on the RPi4 on startup a window
> does not always show up.  If a window does show up it will have only one
> camera then after 10 to 20 seconds the second camera shows up.  The video
> seems frozen at this point but if I move the cameras the image may update
> after a 30 to 60 second or longer delay.

This looks like USB bandwidth issue. This is my guess, assuming 640x480 at 30fps
in YUY2 format, if my math is correct, you'll need 147.5 mb/s per stream. That
would requires 295 mb/s for the raw data, considering you need to add the UVC
header overhead. Also, the RPi USB driver is unlikely capable to reach the
theorical maximum of 480 mb/s.

What I'd try is to connect the cameras on the two USB 3.0 port, or one on each
type. If that still does not work, then perhaps you cameras support JPEG
compression, if so, request JPEG and decode in software.

> 
> I have looked at the gst-launch-1.0 process and threads on both the RPi4 and
> the PC VM.  As expected they look mostly the same but with one exception. 
> There seems to be a large delta time on the 2 queues. Also the pipeline
> images created from the .dot files show no activity on one of the queues. 
> See Images.
> 
> Thanks in advance for any assistance with this problem.
> 
> <
> http://gstreamer-devel.966125.n4.nabble.com/file/t379857/GStreamer_PC_Threads.png
> > 
> 
> <
> http://gstreamer-devel.966125.n4.nabble.com/file/t379857/GStreamer_RPi_Threads.png
> > 
> 
> <http://gstreamer-devel.966125.n4.nabble.com/file/t379857/Queue.png> 
> 
> 
> 
> 
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel