RSPI 4 and Cam streaming

[Nicolas Dufresne]

Le vendredi 28 août 2020 à 05:39 -0500, DejanCotra a écrit :
> hi Nicolas,
> 
> Thanks for response I will check out things that you mentioned.
> 
> I have one more question if you maybe have some hint it would be great.
> 
> Im using RSPi 4 and Gstremer 1.16.2 to stream mic and camera over webrtc.
> 
> Other peer is Windows 10 java application with Gstremer 1.16.2 bindings.
> 
> So when I use pipeline with x264enc :
> v4l2src device=/dev/video0 ! video/x-raw, format=(string)YUY2,
> width=(int)640, height=(int)480 ! videoconvert ! queue ! x264enc bitrate=600
> speed-preset=ultrafast tune=zerolatency key-int-max=15 !
> video/x-h264,profile=constrained-baseline ! h264parse ! rtph264pay
> config-interval=-1 name=payloader ! queue ! " RTP_CAPS_H264 "96
> 
> appsrc name=audiosrc format=3 is-live=1 do-timestamp=1
> caps=audio/x-raw,channels=1,rate=16000,format=S16LE,layout=interleaved !
> audioconvert ! audioresample ! queue ! opusenc ! rtpopuspay ! queue ! "
> RTP_CAPS_OPUS "97
> 
> In my Windows app I can see video and hear sound just fine.
> 
> however when I change video part of pipeline to use HW encoder:
> v4l2src device=/dev/video0 ! video/x-raw, format=(string)YUY2,
> width=(int)640, height=(int)480 ! queue ! v4l2convert ! queue ! v4l2h264enc
> ! video/x-h264,profile=constrained-baseline ! h264parse ! rtph264pay
> config-interval=-1 name=payloader ! queue ! " RTP_CAPS_H264 "96
> 
> So basically I change videoconvert to v4l2convert and x264enc to
> v4l2h264enc. In my Windows app I never receive incoming stream for video,
> audio works just fine.
> I even try to play around with extra-controls for v4l2h264enc
> (extra-controls=s,video_bitrate=600,h264_i_frame_period=15,h264_profile=1,h264_level=10)
> but nothing seems to help.

And no error or warning show up ? Try setting env GST_DEBUG=3, just as
a start. Note that this specific combination is a bit early firmware
and kernel wise (remember none of these drivers have been submitted
upstream yet, and likely for a reason). There is a bit of performance
hit in there, since by default there is a copy between v4l2convert and
v4l2h264enc (this is using contiguous memory with CPU cache disable, so
the copy is very very slow). Assuming the driver supports one or
another, you may want:

  v4l2convert capture-io-mode=dmabuf-import ! queue ! v4l2h264enc ! ...
  v4l2convert ! queue ! v4l2h264enc output-io-mode=dmabuf-import ! ...

One day this will become automatic, but this isn't as simple as it
looks like, specially due to various kernel API limitations and driver
bugs.

p.s. profile and level will be set automatically from the caps
negotiation, no need to use extra-controls for that, video_bitrate is
in bits per second, so should be 600000. Also, you don't want to set
the h264_i_frame_period but the video_gop_size. Quite subtle
difference, but documented:

https://www.kernel.org/doc/html/v5.6/media/uapi/v4l/ext-ctrls-codec.html

That may not solve all your problems of course, but it should in theory
avoid combining multiple bugs together. I got a Pi4 here now, perhaps
you could point me to the system image you are running, I'll be able to
give it a try.

> 
> Dejan
> 
> 
> 
> 
> 
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel