Adapting H264 encoding rate

Thu Jun 8 19:06:49 UTC 2023

Guillaume,

pipeline do not have sufficient buffer to hold audio data when x264 encoder is processing rc-lookahead=20. Add queue element in both audio and video branch to avoid video freeze.
use pass property of x264 to set pass1 encoding, if any problem revert it to cbr. 

Sent from Yahoo Mail. Get the app 

    On Wednesday, 7 June, 2023 at 04:58:44 am GMT-5, Guillaume Denis via gstreamer-devel <gstreamer-devel at lists.freedesktop.org> wrote:  

 Hello,
First of all many thanks for your various suggestions, that helped a lot!
Here is a little followup,
I've been able using mp4mux with varying caps/resolution, with the help of h264timestamper (to prevent this error: "gst_qt_mux_add_buffer:<dry_recorder> error: Buffer has no PTS"), I also switched to h264 main profile and removed zerolatency.
I think your feedback "2 to 3 second IDR frame is good idea for general streaming" is the default value for nvh264enc here: https://gstreamer.freedesktop.org/documentation/nvcodec/GstNvBaseEnc.html?gi-language=c#GstNvBaseEnc:gop-size
What I could not do is:   
   - enable rc-lookahead (20): video freezes and does not recover
   - I could not find how to enable CVBR mode for the encoder
In our scenario, we both record input streams and try to send them back modified (with a video effect) with as little latency as possible over webrtc. Controlling the encoder bitrate is helpful when network conditions vary, but we've lowered the bitrate update period (from 1 to 8 seconds). Anyway, it seems the encoder does not respect the max/target bitrate, especially when the limit is already high.
Thanks again for your leads,Guillaume
Le ven. 2 juin 2023 à 20:18, vinod kesti <vinodkesti at yahoo.com> a écrit :

x264 supports dynamic bitrate but I am not sure about the nvenc. 

Here are my thoughts:1) Updating bitrate every 1 second is bad idea. Use CVBR instead of VBR so that you have better control over the datarate and quality.2) 2 to 3 second IDR frame is good idea for general streaming. May be you can increase if bitrate is too low. In case of 
3) I would suggest control resolution/framerate based on the based bitrate you are choosing. 
By the way latest mp4 muxer supports the dynamic resolution change recording. You can consider using so the dry recording happens without any transcoding.
I am wondering why you are using zero latency and baseline for recording, You can go with rc-lookahead and main profile (Which is supported by most of the devices in todays world) so that you have better quality for same bitrate

Sent from Yahoo Mail. Get the app 

    On Thursday, 1 June, 2023 at 02:18:19 am GMT-5, Guillaume Denis via gstreamer-devel <gstreamer-devel at lists.freedesktop.org> wrote:  

 Hello,
We are using GStreamer within a WebRTC app (the WebRTC part is done with https://github.com/pion/webrtc) where we estimate the best target bitrate depending on network conditions (congestion control).
I was wondering if anyone has experience about the best way to control encoders (x264enc and nvcodec' nvh264enc) in particular:   
   - when we update the target bitrate: should it be done at a slow pace? (or: is updating every second could be useless or even harmful in any way)
   - can it have an impact on keyframes (other than their quality)?
   - are there other encoders' properties worth controlling?
Just in case anyone is interested, below is a typical pipeline we use.
Thanks,Guillaume
## Pipeline with RTP in/out (appsrc/appsink)## and two filesinks (one for the in/dry stream, one after applying a video FX, here facedetect)## we force streams to a 800x600 resolution not to crash muxers with changing resolutions
appsrc name=video_src is-live=true format=GST_FORMAT_TIME min-latency=33333333
appsink name=video_sink qos=true
matroskamux name=dry_recorder ! filesink location=data/dry.mkv
matroskamux name=wet_recorder ! filesink location=data/wet.mkv

video_src. !
application/x-rtp,encoding-name=H264 !
rtpjitterbuffer name=video_buffer do-lost=1 add-reference-timestamp-meta=true latency=200 !
rtph264depay wait-for-keyframe=true !
h264parse !
nvh264sldec min-force-key-unit-interval=3000000000 discard-corrupted-frames=true qos=true !
cudaupload !
cudaconvertscale !
cudadownload !
videorate !
video/x-raw, framerate=30/1, width=800, height=600, format=I420, colorimetry=bt601, chroma-site=jpeg, pixel-aspect-ratio=1/1 !

tee name=tee_video_in !
queue !
nvh264enc name=video_encoder_dry rc-mode=3 preset=4 gop-size=15 zerolatency=true b-adapt=0 bframes=0 rc-lookahead=0 min-force-key-unit-interval=3000000000 qos=true ! video/x-h264, profile=constrained-baseline !
h264parse !
dry_recorder.

tee_video_in. !
queue !
videoconvert !
facedetect !
queue !
cudaupload !
cudaconvertscale !
cudadownload !
video/x-raw, format=I420, colorimetry=bt601, chroma-site=jpeg, pixel-aspect-ratio=1/1 !
nvh264enc name=video_encoder_wet rc-mode=3 preset=4 gop-size=15 zerolatency=true b-adapt=0 bframes=0 rc-lookahead=0 min-force-key-unit-interval=3000000000 qos=true !
video/x-h264, profile=constrained-baseline !
h264parse !

tee name=tee_video_out !
queue !
wet_recorder.

tee_video_out. !
queue !
rtph264pay config-interval=-1 min-ptime=30000000 !
video_sink.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20230608/a0fbe699/attachment-0001.htm>