Issues Using GStreamer with v4l2 Elements

Mon Aug 22 13:33:46 UTC 2022

Le lundi 22 août 2022 à 09:13 -0400, Joshua Quesenberry a écrit :
> Is anyone able to assist me with overcoming this issue? I’m currently unable
> to use a particular make/model of camera (ELP-USBFHD01M w/ OV2710) with the
> v4l2jpegdec element. I’ve had other cameras work with it. I don’t notice any
> errors when upping the GST_DEBUG options. Traffic is at the input pad of the
> element, but nothing coming from the output pad. I have some logs posted on
> the RPi forum issue I posted:
> https://forums.raspberrypi.com/viewtopic.php?p=2031632#p2031632

Perhaps you should still share the pipeline (in gst-launch syntax) that you are
trying to use ? As you don't see any errors, perhaps you should store to disk a
snippet of JPEG and share that too ? As it is now, I don't see much I can add to
help you.

>  
> Thanks,
>  
> Josh Q
>  
> From: Joshua Quesenberry <engnfrc at gmail.com> 
> Sent: Sunday, August 14, 2022 9:00 AM
> To: Nicolas Dufresne <nicolas at ndufresne.ca>
> Cc: Discussion of the development of and with GStreamer
> <gstreamer-devel at lists.freedesktop.org>; Joshua Quesenberry
> <EngnFrc at gmail.com>
> Subject: Re: Issues Using GStreamer with v4l2 Elements
>  
> I’m coming back to this finally. I’ve yet to find a solution to v4l2jpegdec
> not working with this specific make/model camera. It works fine with jpegdec.
> With v4l2jpegdec there’s no downstream data, observable using identity
> element, but I do see data upstream using identity. I’m not seeing any errors
> in dmesg or in the gstreamer output if I up the debug levels. What else can be
> done to try and figure out why things aren’t working? 
>  
> Thanks,
>  
> Josh Q
>  
> On Fri, May 13, 2022 at 4:46 PM Nicolas Dufresne <nicolas at ndufresne.ca> wrote:
> > Le vendredi 13 mai 2022 à 14:28 -0400, Joshua Quesenberry a écrit :
> > > That seems very confusingly labeled, am I missing something in the below
> > > high-level description? From an element's perspective, I wouldn't expect
> > > to be outputting to my sink pad (upstream) or capturing from my src pad
> > > (downstream).
> > > 
> > > Src element -> v4l2 sink using output-io-mode ... v4l2 src using capture-
> > > io-mode -> Sink element
> > > 
> > > In this example, should the elements dmabuf-import be on the upstream or
> > > downstream side? Likewise with the dmabuf? I'm not certain if A) the
> > > downstream is creating the dmabuf and the upstream is importing it to dump
> > > to
> > > or B) the upstream is creating the dmabuf and the downstream is importing
> > > it
> > > for use... it seems like maybe both work when I try?
> > 
> > I don't see a pipeline description I could comment on. By default, all v4l2
> > srcpad are dmabuf exporters. But the sink pad are not automatically
> > importer, as
> > this code is experimental and need some polishing and testing.
> > 
> > > 
> > > Thanks!
> > > 
> > > JQ
> > > 
> > > -----Original Message-----
> > > From: Nicolas Dufresne <nicolas at ndufresne.ca> 
> > > Sent: Friday, May 13, 2022 8:47 AM
> > > To: Joshua Quesenberry <engnfrc at gmail.com>
> > > Cc: Discussion of the development of and with GStreamer
> > > <gstreamer-devel at lists.freedesktop.org>
> > > Subject: Re: Issues Using GStreamer with v4l2 Elements
> > > 
> > > Le vendredi 13 mai 2022 à 00:52 -0400, Joshua Quesenberry a écrit :
> > > > Hi Nicolas,
> > > > 
> > > > Sorry for the late reply, I got covered up with other stuff and am 
> > > > just getting a minute to come back to this.
> > > > 
> > > > > Just read your post, looks like you are asking about at least two 
> > > > > distinct issue. In one case, v4l2jpegdec does not work at all for 
> > > > > you. There isn't much information, have you considering prepending 
> > > > > jpegparse in front of it ? That may help ? Jpeg V4L2 decoder is 
> > > > > probably one of the least tested decoder in the v4l2 lot.
> > > > 
> > > > Correct, two separate issues I'm running into.
> > > > 
> > > > v4l2jpegdec works fine with one camera, but not the other. I'm setting 
> > > > them both up very similarly, so it escapes me what could be different 
> > > > to cause this behavior. I just tried jpegparse in the pipeline just 
> > > > before v4l2jpegdec and that didn't have any effect, still the identity 
> > > > just after the decode is never printed. I also tried dropping down 
> > > > from 1280x720 at 60fps to 1024x768 at 30fps to lessen the load a bit and 
> > > > that had no effect either.
> > > > 
> > > > > In the second issue, it works, but its slow. Could it be that you 
> > > > > are loading the CPU ? v4l2h264enc does not automatically do 
> > > > > zero-copy. You can experimentally enable it passing output-io-
> > > > > mode=dmabuf-import.
> > > > 
> > > > I suppose it could be GPU load? When I've looked for ways to measure 
> > > > what the GPU is doing, I've unfortunately not found much of 
> > > > significance. The CPU load is quite low during this testing, <10% for 
> > > > the gstreamer pipeline according to htop. I have a pipeline already 
> > > > with the output-io-modes set to dmabuf-import, but before my original 
> > > > post I switched everything back to auto in hopes things may start 
> > > > working appropriately, however that was not the case. Keep in mind 
> > > > that I have an identity in between the source caps and the capssetter, 
> > > > this is where I see the average framerate drop with v4l2h264enc 
> > > > present.
> > > > 
> > > > I've found that by adding more queues to the pipeline and then setting 
> > > > all queues to have a leaky downstream that the average rate at the 
> > > > source has now climbed back up to what I expected all along, so I'm 
> > > > assuming the issue is a clogged up downstream affecting the upstream 
> > > > as far as the source? After these changes input rate is ~30fps and 
> > > > output to file ~15fps, which is what I'm asking for, and the CPU usage 
> > > > is around ~30%. Playing around with the dmabuf-import option it 
> > > > appears I'm able to get similar rates with a drop in CPU usage to 
> > > > around ~10%, very exciting!
> > > > 
> > > > How do the v4l2 elements work with queues? What do they do when stuck 
> > > > in between a dmabuf-import src pad and dmabuf sink pad? They don't 
> > > > have io-modes to be specified (at least at the user level) and I don't 
> > > > see a v4l2queue or something of the like. Tomorrow I'll do some more 
> > > > experiments to see which queues that I've added are unnecessary.
> > > 
> > > M2M elements have two queues, so you have output-io-mode (the sink pad)
> > > and capture-io-mode (for the src pad).
> > > 
> > > > 
> > > > So now, camera #1 not getting through v4l2jpegdec is my major 
> > > > roadblock. Anyone have ideas on how to debug this one or know what 
> > > > could cause this behavior?
> > > > 
> > > > FYI, I did dump additional information to the RPi Forums question a 
> > > > few days ago, if something else is needed, just let me know.
> > > > 
> > > > Thanks!
> > > > 
> > > > JQ
> > > > 
> > > > On Mon, May 9, 2022 at 4:15 PM Nicolas Dufresne <nicolas at ndufresne.ca>
> > > > wrote:
> > > > > 
> > > > > Le lundi 09 mai 2022 à 12:04 -0400, Joshua Quesenberry via 
> > > > > gstreamer-devel a écrit :
> > > > > > Hey Everyone!
> > > > > > 
> > > > > > I’m having some trouble using the v4l2 elements on an RPi. I’ve 
> > > > > > posted a detailed summary on the Raspberry Pi forum. Can anyone 
> > > > > > lend a hand? I’m not sure how many of you are frequent users of 
> > > > > > the RPi, so figured I’d reach out here in parallel in case the RPi
> > > > > > forum isn’t the right audience.
> > > > > > https://forums.raspberrypi.com/viewtopic.php?t=334215
> > > > > 
> > > > > Just read your post, looks like you are asking about at least two 
> > > > > distinct issue. In one case, v4l2jpegdec does not work at all for 
> > > > > you. There isn't much information, have you considering prepending 
> > > > > jpegparse in front of it ? That may help ? Jpeg V4L2 decoder is 
> > > > > probably one of the least tested decoder in the v4l2 lot.
> > > > > 
> > > > > In the second issue, it works, but its slow. Could it be that you 
> > > > > are loading the CPU ? v4l2h264enc does not automatically do 
> > > > > zero-copy. You can experimentally enable it passing output-io-
> > > > > mode=dmabuf-import.
> > > > > 
> > > > > Nicolas
> > > > > 
> > > > > > 
> > > > > > Thanks,
> > > > > > 
> > > > > > JQ
> > > > > > 
> > > > > > 
> > > > > 
> > > 
> > > 