x264enc bitrate adjustment latency

[Samuel Hurst]

Hello,

I've been experimenting with congestion control for RTP, and as part of 
this I have a use case where the "bitrate" property on the encoder in a 
sending pipeline is changed dynamically in response to network conditions.

I've been using the x264enc element in GStreamer as my encoder for this, 
and I've been observing that x264enc seems to take a while to react to 
the "bitrate" property being changed before that actually takes a second 
or two before the encoder actually reacts to a requested reduction in 
bit rate. This is then causing further congestion on the network that 
I'm testing on.

I've taken the congestion controller out of the loop and written a small 
application which adjusts the bitrate property on the x264enc element as 
a stand alone tool [1], with effectively the following pipeline:

videotestsrc horizontal-speed=10 ! \
   video/x-raw,format=NV12,width=1920,height=1080,framerate=25/1 ! \
   x264enc bitrate=10000 speed-preset=ultrafast \
     tune=zerolatency+fastdecode ! \
   queue ! \
   avdec_h264 ! \
   queue ! \
   fpsdisplaysink

I've then been graphing the output to understand the time between the 
bitrate property being set, and the new rate taking effect. I've been 
using the gst-shark [2] bitrate and buffer tracers to track the average 
bit rate out of the x264enc element as well as the individual sizes of 
each encoded frame.

The graph linked in [3] shows the requested bit rate in gold, the 
average bit rate as seen by gst-shark in red and the size of each 
encoded frame in blue. In the example, I've reduced the key-int-max 
parameter to 25, but regardless of the value of this parameter I still 
see the same 1-2 second delay in any attempt by the encoder to conform 
to the bit rate limitation.

Is there any way of reducing the latency between setting the bitrate 
parameter to it actually taking effect? I've so far tried adjusting the
key-int-max, qp-max, qp-step, rc-lookahead and vbv-buf-capacity 
attributes but nothing seems to improve it. I've tried with GStreamer 
1.16.3 and 1.20.2. I'm using libx264 version 163.

Thanks in advance,
-Sam

[1]: https://www.dropbox.com/s/bofyyfcee2m5olz/gst-launch-h264.c?dl=0
[2]: https://developer.ridgerun.com/wiki/index.php?title=GstShark
[3]: 
https://www.dropbox.com/s/d9sxya0mr00j5np/x264-buffer-bitrate-test-idr-25-10mb-to-8mb.png?dl=0