Raspberry Pi 4 is not working when the camera video size is increased.

iotsystek iotsystek at gmail.com
Fri Jan 29 20:58:35 UTC 2021 

Nicolas Dufresne-5 wrote
> Le vendredi 29 janvier 2021 à 11:28 -0600, iotsystek a écrit :
>> I am new to Linux and GStreamer.  I am working on getting 4 - 1 mega
>> pixel
>> USB 2.0 cameras working on a RPi4.  Currently I am using just 2 cameras
>> for
>> testing.  The cameras are connected via USB 2.0 directly to the Pi (no
>> external hub).  The RPi4 has 256Mb GPU memory set and is being powered
>> with
>> a precision bench power supply at 5.05 volts and does not complain about
>> low
>> voltage.  There is no audio in this project.  My final application will
>> be
>> embedded into my diesel truck and 5th wheel.
>> 
>> I have 2 different hardware test configurations:
>> 1       A Raspberry Pi 4 with Buster OS dates Dec 2 2020.
>>         Each camera is connected directly to a Pi USB port.
>>         GStreamer version is 1.14.4
>> 
>> 2       Linux Ubuntu 18.04 running in a virtual machine on my PC.
>>         1 camera is connected directly to a PC USB 3.0 port.
>>         1 camera is connected to a PC via a USB 3.0 hub.
>>         GStreamer version is 1.14.5
>> 
>> The following 2 gst-launch-1.0 commands are being used.  The only
>> difference
>> between them are the size of the video 320x240 or 640x480.
>> 
>> gst-launch-1.0 videomixer name=mix ! videoconvert ! autovideosink
>> sync=false
>> \
>>  v4l2src device=/dev/video0 ! videorate max-rate=6 ! queue ! videoconvert
>> !
>> video/x-raw, format=AYUV, width=320, height=240 ! videobox border-alpha=0
>> top=0    left=0    ! mix. \
>>  v4l2src device=/dev/video2 ! videorate max-rate=6 ! queue ! videoconvert
>> !
>> video/x-raw, format=AYUV, width=320, height=240 ! videobox border-alpha=0
>> top=0    left=-320 ! mix.
>> 
>> gst-launch-1.0 videomixer name=mix ! videoconvert ! ximagesink sync=false
>> \
>>  v4l2src device=/dev/video0 ! videorate max-rate=6 ! queue ! videoconvert
>> !
>> video/x-raw, format=AYUV, width=640, height=480 ! videobox border-alpha=0
>> top=0    left=0    ! mix. \
>>  v4l2src device=/dev/video2 ! videorate max-rate=6 ! queue ! videoconvert
>> !
>> video/x-raw, format=AYUV, width=640, height=480 ! videobox border-alpha=0
>> top=0    left=-640 ! mix.
>> 
>> In the following write up my definition of working means that the video
>> starts up with gst-launch-1.0 and updates as expected when I move the
>> camera
>> around. (Real time video)
>> 
>> Both of the above commands work on the PC VM.
>> 
>> On the RPi4 the smaller window always works but the larger window only
>> kind
>> of works sometimes. With the larger window on the RPi4 on startup a
>> window
>> does not always show up.  If a window does show up it will have only one
>> camera then after 10 to 20 seconds the second camera shows up.  The video
>> seems frozen at this point but if I move the cameras the image may update
>> after a 30 to 60 second or longer delay.
> 
> This looks like USB bandwidth issue. This is my guess, assuming 640x480 at
> 30fps
> in YUY2 format, if my math is correct, you'll need 147.5 mb/s per stream.
> That
> would requires 295 mb/s for the raw data, considering you need to add the
> UVC
> header overhead. Also, the RPi USB driver is unlikely capable to reach the
> theorical maximum of 480 mb/s.
> 
> The Pi is booting from a SSD on a USB 3.0 port, the wireless keyboard and
> mouse are on a USB2.0 port and the 2 cameras are on 1 USB 3.0 and 1 USB2.0
> ports.  I have also tried putting both cameras on a USB 3.0 hub and
> connecting that to the Pi 3.0 port with the same results.
> 
> What I'd try is to connect the cameras on the two USB 3.0 port, or one on
> each
> type. If that still does not work, then perhaps you cameras support JPEG
> compression, if so, request JPEG and decode in software.
> 
> The cameras can output the following formats:
> 
> v4l2-ctl --device /dev/video0 --list-formats-ext
> 
> ioctl: VIDIOC_ENUM_FMT
> 	Type: Video Capture
> 
> 	[0]: 'MJPG' (Motion-JPEG, compressed)
> 		Size: Discrete 1280x720
> 			Interval: Discrete 0.033s (30.000 fps)
> 		Size: Discrete 640x480
> 			Interval: Discrete 0.033s (30.000 fps)
> 		Size: Discrete 352x288
> 			Interval: Discrete 0.033s (30.000 fps)
> 		Size: Discrete 320x240
> 			Interval: Discrete 0.033s (30.000 fps)
> 		Size: Discrete 176x144
> 			Interval: Discrete 0.033s (30.000 fps)
> 		Size: Discrete 160x120
> 			Interval: Discrete 0.033s (30.000 fps)
> 		Size: Discrete 800x600
> 			Interval: Discrete 0.033s (30.000 fps)
> 		Size: Discrete 960x720
> 			Interval: Discrete 0.033s (30.000 fps)
> 	[1]: 'YUYV' (YUYV 4:2:2)
> 		Size: Discrete 1280x720
> 			Interval: Discrete 0.100s (10.000 fps)
> 		Size: Discrete 640x480
> 			Interval: Discrete 0.033s (30.000 fps)
> 		Size: Discrete 352x288
> 			Interval: Discrete 0.033s (30.000 fps)
> 		Size: Discrete 320x240
> 			Interval: Discrete 0.033s (30.000 fps)
> 		Size: Discrete 176x144
> 			Interval: Discrete 0.033s (30.000 fps)
> 		Size: Discrete 160x120
> 			Interval: Discrete 0.033s (30.000 fps)
> 		Size: Discrete 800x600
> 			Interval: Discrete 0.050s (20.000 fps)
> 		Size: Discrete 960x720
> 			Interval: Discrete 0.067s (15.000 fps)
> 
> I have been unable to figure out the syntax to configure v4l2src to use
> MJPG.
> 
> Also the GStreamer web site says the videomixer is limited to only 3
> different inputs, I quote: “Videomixer can accept AYUV, ARGB and BGRA
> video streams”.  I’m not sure which of these 3 I should use for best
> performance but I have found that if I don’t use one of them, then when
> the window is displayed only the last video source specified is displayed
> with the other 3 are blacked out.  Its like the boarders of the last
> window are blocking out the other videos.
> 
>> 
>> I have looked at the gst-launch-1.0 process and threads on both the RPi4
>> and
>> the PC VM.  As expected they look mostly the same but with one exception. 
>> There seems to be a large delta time on the 2 queues. Also the pipeline
>> images created from the .dot files show no activity on one of the queues. 
>> See Images.
>> 
>> Thanks in advance for any assistance with this problem.
>> 
>> <
>> http://gstreamer-devel.966125.n4.nabble.com/file/t379857/GStreamer_PC_Threads.png
>> > 
>> 
>> <
>> http://gstreamer-devel.966125.n4.nabble.com/file/t379857/GStreamer_RPi_Threads.png
>> > 
>> 
>> <http://gstreamer-devel.966125.n4.nabble.com/file/t379857/Queue.png> 
>> 
>> 
>> 
>> 
>> --
>> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
>> _______________________________________________
>> gstreamer-devel mailing list
>> 

> gstreamer-devel at .freedesktop

>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> 
> 
> _______________________________________________
> gstreamer-devel mailing list

> gstreamer-devel at .freedesktop

> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel





--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/

More information about the gstreamer-devel mailing list