Issues Using GStreamer with v4l2 Elements

[Joshua Quesenberry]

Hi Nicolas,

Sorry for the late reply, I got covered up with other stuff and am
just getting a minute to come back to this.

> Just read your post, looks like you are asking about at least two distinct
> issue. In one case, v4l2jpegdec does not work at all for you. There isn't much
> information, have you considering prepending jpegparse in front of it ? That may
> help ? Jpeg V4L2 decoder is probably one of the least tested decoder in the v4l2
> lot.

Correct, two separate issues I'm running into.

v4l2jpegdec works fine with one camera, but not the other. I'm setting
them both up very similarly, so it escapes me what could be different
to cause this behavior. I just tried jpegparse in the pipeline just
before v4l2jpegdec and that didn't have any effect, still the identity
just after the decode is never printed. I also tried dropping down
from 1280x720 at 60fps to 1024x768 at 30fps to lessen the load a bit and
that had no effect either.

> In the second issue, it works, but its slow. Could it be that you are loading
> the CPU ? v4l2h264enc does not automatically do zero-copy. You can
> experimentally enable it passing output-io-mode=dmabuf-import.

I suppose it could be GPU load? When I've looked for ways to measure
what the GPU is doing, I've unfortunately not found much of
significance. The CPU load is quite low during this testing, <10% for
the gstreamer pipeline according to htop. I have a pipeline already
with the output-io-modes set to dmabuf-import, but before my original
post I switched everything back to auto in hopes things may start
working appropriately, however that was not the case. Keep in mind
that I have an identity in between the source caps and the capssetter,
this is where I see the average framerate drop with v4l2h264enc
present.

I've found that by adding more queues to the pipeline and then setting
all queues to have a leaky downstream that the average rate at the
source has now climbed back up to what I expected all along, so I'm
assuming the issue is a clogged up downstream affecting the upstream
as far as the source? After these changes input rate is ~30fps and
output to file ~15fps, which is what I'm asking for, and the CPU usage
is around ~30%. Playing around with the dmabuf-import option it
appears I'm able to get similar rates with a drop in CPU usage to
around ~10%, very exciting!

How do the v4l2 elements work with queues? What do they do when stuck
in between a dmabuf-import src pad and dmabuf sink pad? They don't
have io-modes to be specified (at least at the user level) and I don't
see a v4l2queue or something of the like. Tomorrow I'll do some more
experiments to see which queues that I've added are unnecessary.

So now, camera #1 not getting through v4l2jpegdec is my major
roadblock. Anyone have ideas on how to debug this one or know what
could cause this behavior?

FYI, I did dump additional information to the RPi Forums question a
few days ago, if something else is needed, just let me know.

Thanks!

JQ

On Mon, May 9, 2022 at 4:15 PM Nicolas Dufresne <nicolas at ndufresne.ca> wrote:
>
> Le lundi 09 mai 2022 à 12:04 -0400, Joshua Quesenberry via gstreamer-devel a
> écrit :
> > Hey Everyone!
> >
> > I’m having some trouble using the v4l2 elements on an RPi. I’ve posted a
> > detailed summary on the Raspberry Pi forum. Can anyone lend a hand? I’m not
> > sure how many of you are frequent users of the RPi, so figured I’d reach out
> > here in parallel in case the RPi forum isn’t the right audience.
> > https://forums.raspberrypi.com/viewtopic.php?t=334215
>
> Just read your post, looks like you are asking about at least two distinct
> issue. In one case, v4l2jpegdec does not work at all for you. There isn't much
> information, have you considering prepending jpegparse in front of it ? That may
> help ? Jpeg V4L2 decoder is probably one of the least tested decoder in the v4l2
> lot.
>
> In the second issue, it works, but its slow. Could it be that you are loading
> the CPU ? v4l2h264enc does not automatically do zero-copy. You can
> experimentally enable it passing output-io-mode=dmabuf-import.
>
> Nicolas
>
> >
> > Thanks,
> >
> > JQ
> >
> >
>