Issues Using GStreamer with v4l2 Elements
Joshua Quesenberry
engnfrc at gmail.com
Tue Aug 23 19:46:24 UTC 2022
Nicolas,
I've created an Issue (https://github.com/raspberrypi/linux/issues/5142) on the raspberrypi/linux repo. If you have an account there I wasn't able to @ it. 6by9 ran the jpeg through JPEGSnoop and it's complaining about bad Huffman codes and overreading scan segments. Any thoughts on that? Would love to get all the minds talking in one place, so if you think it'd help and are willing to chime in on the Issue I've generated that's be awesome.
Thanks!
Josh Q
-----Original Message-----
From: Nicolas Dufresne <nicolas at ndufresne.ca>
Sent: Monday, August 22, 2022 2:18 PM
To: Joshua Quesenberry <engnfrc at gmail.com>
Cc: 'Discussion of the development of and with GStreamer' <gstreamer-devel at lists.freedesktop.org>
Subject: Re: Issues Using GStreamer with v4l2 Elements
Hi Josh,
Le lundi 22 août 2022 à 13:23 -0400, Joshua Quesenberry a écrit :
> Thanks Nicolas. I've forwarded that information over to my RPi forums post. I don't suppose you have a ticket # available that I can reference?
Can't recall exactly, since a lot of the information got shared to me through chats (LibreELEC Slack and GSTreamer IRC) while trying to get this bigger issue
sorted:
https://github.com/raspberrypi/firmware/issues/1673
I know a lot of firmware fixes were being shipped in parallel to this series of backport.
>
> Thanks,
>
> Josh Q
>
> -----Original Message-----
> From: Nicolas Dufresne <nicolas at ndufresne.ca>
> Sent: Monday, August 22, 2022 1:09 PM
> To: Joshua Quesenberry <engnfrc at gmail.com>
> Cc: 'Discussion of the development of and with GStreamer'
> <gstreamer-devel at lists.freedesktop.org>
> Subject: Re: Issues Using GStreamer with v4l2 Elements
>
> Hi Joshua,
>
> Le lundi 22 août 2022 à 10:58 -0400, Joshua Quesenberry a écrit :
> > Nicolas,
> >
> > Please find attached zip files for a jpeg capture and a raw capture.
>
> From a quick look, my impression is that the driver does delay the end of the draining flow (CMD_STOP or bytesused = 0) to after the last decoded buffer has been dequeued. From your trace, you can observe this by a chain from the identity_pre and not for the identify_post.
>
> I believe this is an issue that was previously reported to RPi maintainers and should in theory be known or addressed.
>
> regards,
> Nicolas
>
>
> >
> > Thanks!
> >
> > Josh Q
> >
> > -----Original Message-----
> > From: Nicolas Dufresne <nicolas at ndufresne.ca>
> > Sent: Monday, August 22, 2022 9:34 AM
> > To: Joshua Quesenberry <engnfrc at gmail.com>
> > Cc: 'Discussion of the development of and with GStreamer'
> > <gstreamer-devel at lists.freedesktop.org>
> > Subject: Re: Issues Using GStreamer with v4l2 Elements
> >
> > Le lundi 22 août 2022 à 09:13 -0400, Joshua Quesenberry a écrit :
> > > Is anyone able to assist me with overcoming this issue? I’m
> > > currently unable to use a particular make/model of camera
> > > (ELP-USBFHD01M w/
> > > OV2710) with the v4l2jpegdec element. I’ve had other cameras work
> > > with it. I don’t notice any errors when upping the GST_DEBUG options.
> > > Traffic is at the input pad of the element, but nothing coming
> > > from the output pad. I have some logs posted on the RPi forum issue I posted:
> > > https://forums.raspberrypi.com/viewtopic.php?p=2031632#p2031632
> >
> > Perhaps you should still share the pipeline (in gst-launch syntax) that you are trying to use ? As you don't see any errors, perhaps you should store to disk a snippet of JPEG and share that too ? As it is now, I don't see much I can add to help you.
> >
> > >
> > > Thanks,
> > >
> > > Josh Q
> > >
> > > From: Joshua Quesenberry <engnfrc at gmail.com>
> > > Sent: Sunday, August 14, 2022 9:00 AM
> > > To: Nicolas Dufresne <nicolas at ndufresne.ca>
> > > Cc: Discussion of the development of and with GStreamer
> > > <gstreamer-devel at lists.freedesktop.org>; Joshua Quesenberry
> > > <EngnFrc at gmail.com>
> > > Subject: Re: Issues Using GStreamer with v4l2 Elements
> > >
> > > I’m coming back to this finally. I’ve yet to find a solution to
> > > v4l2jpegdec not working with this specific make/model camera. It works fine with jpegdec.
> > > With v4l2jpegdec there’s no downstream data, observable using
> > > identity element, but I do see data upstream using identity. I’m
> > > not seeing any errors in dmesg or in the gstreamer output if I up the debug levels.
> > > What else can be done to try and figure out why things aren’t working?
> > >
> > > Thanks,
> > >
> > > Josh Q
> > >
> > > On Fri, May 13, 2022 at 4:46 PM Nicolas Dufresne <nicolas at ndufresne.ca> wrote:
> > > > Le vendredi 13 mai 2022 à 14:28 -0400, Joshua Quesenberry a écrit :
> > > > > That seems very confusingly labeled, am I missing something in
> > > > > the below high-level description? From an element's
> > > > > perspective, I wouldn't expect to be outputting to my sink pad
> > > > > (upstream) or capturing from my src pad (downstream).
> > > > >
> > > > > Src element -> v4l2 sink using output-io-mode ... v4l2 src
> > > > > using
> > > > > capture- io-mode -> Sink element
> > > > >
> > > > > In this example, should the elements dmabuf-import be on the
> > > > > upstream or downstream side? Likewise with the dmabuf? I'm not
> > > > > certain if A) the downstream is creating the dmabuf and the
> > > > > upstream is importing it to dump to or B) the upstream is
> > > > > creating the dmabuf and the downstream is importing it for
> > > > > use... it seems like maybe both work when I try?
> > > >
> > > > I don't see a pipeline description I could comment on. By
> > > > default, all v4l2 srcpad are dmabuf exporters. But the sink pad
> > > > are not automatically importer, as this code is experimental and
> > > > need some polishing and testing.
> > > >
> > > > >
> > > > > Thanks!
> > > > >
> > > > > JQ
> > > > >
> > > > > -----Original Message-----
> > > > > From: Nicolas Dufresne <nicolas at ndufresne.ca>
> > > > > Sent: Friday, May 13, 2022 8:47 AM
> > > > > To: Joshua Quesenberry <engnfrc at gmail.com>
> > > > > Cc: Discussion of the development of and with GStreamer
> > > > > <gstreamer-devel at lists.freedesktop.org>
> > > > > Subject: Re: Issues Using GStreamer with v4l2 Elements
> > > > >
> > > > > Le vendredi 13 mai 2022 à 00:52 -0400, Joshua Quesenberry a écrit :
> > > > > > Hi Nicolas,
> > > > > >
> > > > > > Sorry for the late reply, I got covered up with other stuff
> > > > > > and am just getting a minute to come back to this.
> > > > > >
> > > > > > > Just read your post, looks like you are asking about at
> > > > > > > least two distinct issue. In one case, v4l2jpegdec does
> > > > > > > not work at all for you. There isn't much information,
> > > > > > > have you considering prepending jpegparse in front of it ?
> > > > > > > That may help ? Jpeg V4L2 decoder is probably one of the least tested decoder in the v4l2 lot.
> > > > > >
> > > > > > Correct, two separate issues I'm running into.
> > > > > >
> > > > > > v4l2jpegdec works fine with one camera, but not the other.
> > > > > > I'm setting them both up very similarly, so it escapes me
> > > > > > what could be different to cause this behavior. I just tried
> > > > > > jpegparse in the pipeline just before v4l2jpegdec and that
> > > > > > didn't have any effect, still the identity just after the
> > > > > > decode is never printed. I also tried dropping down from
> > > > > > 1280x720 at 60fps to 1024x768 at 30fps to lessen the load a bit and that had no effect either.
> > > > > >
> > > > > > > In the second issue, it works, but its slow. Could it be
> > > > > > > that you are loading the CPU ? v4l2h264enc does not
> > > > > > > automatically do zero-copy. You can experimentally enable
> > > > > > > it passing
> > > > > > > output-io- mode=dmabuf-import.
> > > > > >
> > > > > > I suppose it could be GPU load? When I've looked for ways to
> > > > > > measure what the GPU is doing, I've unfortunately not found
> > > > > > much of significance. The CPU load is quite low during this
> > > > > > testing, <10% for the gstreamer pipeline according to htop.
> > > > > > I have a pipeline already with the output-io-modes set to
> > > > > > dmabuf-import, but before my original post I switched
> > > > > > everything back to auto in hopes things may start working
> > > > > > appropriately, however that was not the case. Keep in mind
> > > > > > that I have an identity in between the source caps and the
> > > > > > capssetter, this is where I see the average framerate drop with v4l2h264enc present.
> > > > > >
> > > > > > I've found that by adding more queues to the pipeline and
> > > > > > then setting all queues to have a leaky downstream that the
> > > > > > average rate at the source has now climbed back up to what I
> > > > > > expected all along, so I'm assuming the issue is a clogged
> > > > > > up downstream affecting the upstream as far as the source?
> > > > > > After these changes input rate is ~30fps and output to file
> > > > > > ~15fps, which is what I'm asking for, and the CPU usage is
> > > > > > around ~30%. Playing around with the dmabuf-import option it
> > > > > > appears I'm able to get similar rates with a drop in CPU usage to around ~10%, very exciting!
> > > > > >
> > > > > > How do the v4l2 elements work with queues? What do they do
> > > > > > when stuck in between a dmabuf-import src pad and dmabuf sink pad?
> > > > > > They don't have io-modes to be specified (at least at the
> > > > > > user
> > > > > > level) and I don't see a v4l2queue or something of the like.
> > > > > > Tomorrow I'll do some more experiments to see which queues that I've added are unnecessary.
> > > > >
> > > > > M2M elements have two queues, so you have output-io-mode (the
> > > > > sink
> > > > > pad) and capture-io-mode (for the src pad).
> > > > >
> > > > > >
> > > > > > So now, camera #1 not getting through v4l2jpegdec is my
> > > > > > major roadblock. Anyone have ideas on how to debug this one
> > > > > > or know what could cause this behavior?
> > > > > >
> > > > > > FYI, I did dump additional information to the RPi Forums
> > > > > > question a few days ago, if something else is needed, just let me know.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > JQ
> > > > > >
> > > > > > On Mon, May 9, 2022 at 4:15 PM Nicolas Dufresne
> > > > > > <nicolas at ndufresne.ca>
> > > > > > wrote:
> > > > > > >
> > > > > > > Le lundi 09 mai 2022 à 12:04 -0400, Joshua Quesenberry via
> > > > > > > gstreamer-devel a écrit :
> > > > > > > > Hey Everyone!
> > > > > > > >
> > > > > > > > I’m having some trouble using the v4l2 elements on an RPi.
> > > > > > > > I’ve posted a detailed summary on the Raspberry Pi forum.
> > > > > > > > Can anyone lend a hand? I’m not sure how many of you are
> > > > > > > > frequent users of the RPi, so figured I’d reach out here
> > > > > > > > in parallel in case the RPi forum isn’t the right audience.
> > > > > > > > https://forums.raspberrypi.com/viewtopic.php?t=334215
> > > > > > >
> > > > > > > Just read your post, looks like you are asking about at
> > > > > > > least two distinct issue. In one case, v4l2jpegdec does
> > > > > > > not work at all for you. There isn't much information,
> > > > > > > have you considering prepending jpegparse in front of it ?
> > > > > > > That may help ? Jpeg V4L2 decoder is probably one of the least tested decoder in the v4l2 lot.
> > > > > > >
> > > > > > > In the second issue, it works, but its slow. Could it be
> > > > > > > that you are loading the CPU ? v4l2h264enc does not
> > > > > > > automatically do zero-copy. You can experimentally enable
> > > > > > > it passing
> > > > > > > output-io- mode=dmabuf-import.
> > > > > > >
> > > > > > > Nicolas
> > > > > > >
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > JQ
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > > >
> >
>
>
More information about the gstreamer-devel
mailing list