x264enc bitrate adjustment latency

[Nicolas Dufresne]

Le mercredi 11 mai 2022 à 14:43 +0100, Samuel Hurst via gstreamer-devel a
écrit :
> Hello,
> 
> I've been experimenting with congestion control for RTP, and as part of 
> this I have a use case where the "bitrate" property on the encoder in a 
> sending pipeline is changed dynamically in response to network conditions.
> 
> I've been using the x264enc element in GStreamer as my encoder for this, 
> and I've been observing that x264enc seems to take a while to react to 
> the "bitrate" property being changed before that actually takes a second 
> or two before the encoder actually reacts to a requested reduction in 
> bit rate. This is then causing further congestion on the network that 
> I'm testing on.
> 
> I've taken the congestion controller out of the loop and written a small 
> application which adjusts the bitrate property on the x264enc element as 
> a stand alone tool [1], with effectively the following pipeline:
> 
> videotestsrc horizontal-speed=10 ! \
>    video/x-raw,format=NV12,width=1920,height=1080,framerate=25/1 ! \
>    x264enc bitrate=10000 speed-preset=ultrafast \
>      tune=zerolatency+fastdecode ! \
>    queue ! \

This queue will fill in, causing high latency as you observe. Sets is-live=1 on
videotestsrc, configure that queue to be smaller.

>    avdec_h264 ! \
>    queue ! \
>    fpsdisplaysink
> 
> I've then been graphing the output to understand the time between the 
> bitrate property being set, and the new rate taking effect. I've been 
> using the gst-shark [2] bitrate and buffer tracers to track the average 
> bit rate out of the x264enc element as well as the individual sizes of 
> each encoded frame.
> 
> The graph linked in [3] shows the requested bit rate in gold, the 
> average bit rate as seen by gst-shark in red and the size of each 
> encoded frame in blue. In the example, I've reduced the key-int-max 
> parameter to 25, but regardless of the value of this parameter I still 
> see the same 1-2 second delay in any attempt by the encoder to conform 
> to the bit rate limitation.
> 
> Is there any way of reducing the latency between setting the bitrate 
> parameter to it actually taking effect? I've so far tried adjusting the
> key-int-max, qp-max, qp-step, rc-lookahead and vbv-buf-capacity 
> attributes but nothing seems to improve it. I've tried with GStreamer 
> 1.16.3 and 1.20.2. I'm using libx264 version 163.
> 
> Thanks in advance,
> -Sam
> 
> [1]: https://www.dropbox.com/s/bofyyfcee2m5olz/gst-launch-h264.c?dl=0
> [2]: https://developer.ridgerun.com/wiki/index.php?title=GstShark
> [3]: 
> https://www.dropbox.com/s/d9sxya0mr00j5np/x264-buffer-bitrate-test-idr-25-10mb-to-8mb.png?dl=0