[Spice-devel] [spice v2] streaming: Always delegate bit rate control to the video encoder

Thu Oct 27 18:38:54 UTC 2016

Here are some pointers on ways to test this patch.

1. Tweaking the available bandwith at will
-------------------------------------------

The point of the bit rate control algorithm is to adapt the video 
stream's bitrate to the available network bandwidth. So you could test 
how it reacts in various conditions by moving closer or further from 
your wireless access point, switching to an ethernet connection, or 
setting up port fowarding to send the stream around hosts on the 
internet. But to get reproducible conditions the best is to use tc.

Here is an example of the commands I use:
  To limit the Spice server->client traffic and not the client->server one:
  tc qdisc add dev lo handle 1:0 root htb direct_qlen 10
  tc class add dev lo parent 1:0 classid 1:11 htb rate 6000kbit #direct_qlen 10
  # Optionally increase latency. Note that a high latency will limit bandwidth.
  # With 150ms we can still achieve >> 20Mbps.
  tc qdisc add dev lo parent 1:11 netem delay 150ms 10ms
  tc filter add dev lo protocol ip prio 1 u32 match ip sport 5903 0xffff flowid 1:11
  # Prevent too much packet queueing, otherwise packets could arrive seconds
  # after being sent which is not realistic and broken (see Bufferbloat).
  ifconfig lo txqueuelen 1

The above assumes the server listens on port 5903 on localhost. It 
limits the server -> client bandwidth to 6 Mbps with a 150 ms RTT (150 
ms in one direction, essentially 0 ms in the other) with 10ms of jitter.

Initially the client will totally mis-estimate the available bandwidth, 
sometimes coming up with > 20 Gbps. So it may take a dozen seconds for 
the server to drop the bitrate low enough to get a working stream.

Then you can drop the available bitrate to see how it reacts:

  tc class change dev lo parent 1:0 classid 1:11 htb rate 4000kbit

Finally you can check the configuration and whether packets are treated as expected:
  tc -s qdisc ls dev lo
  tc -s class show dev lo
  tc -s filter show dev lo

And clear everything once you've done all your tests.
  tc qdisc del dev lo root

For more details see:
  http://www.insightfullogic.com/blog/2013/jul/25/performance-tests-slow-networks-tc/
  http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm

2. Testing the current "server-drops-only" bit rate control
-----------------------------------------------------------

It's a good idea to play with the above commands to get an idea of how 
the baseline behaves before modifying the code. If you've done so, 
you've tested the bitrate control of either the MJPEG or the GStreamer 
video encoder.

But what we want to test here is the bitrate control implemented in 
stream.c and dcc-send.c. But it is only used when two conditions are 
met:

 * dcc_use_video_encoder_rate_control() must return false, which means 
   the client cannot send stream report messages (to summarize these 
   tell the server how much margin there is at the client between when a 
   frame is received and when it must be displayed; the bigger the 
   frame, the longer it spends in transit and the shorter that margin). 
   This is essnetially for compatibility with old clients (like the 
   defunct spicec).

 * dcc_is_low_bandwidth() must return true, which means the initial
   bandwidth estimate was under 10 Mbps. This can be a problem for the 
   tests as, at least when using tc, the initial bandwidth estimate can 
   be wildly off (like >> 20Gbps when the bandwidth is capped at 6Mbps).

This gives us three cases:
 * use_video_encoder_rate_control == true
   -> Then it is the video encoder that adjusts the stream's bitrate.

 * use_video_encoder_rate_control == false && 
   is_low_bandwidth == true
   -> Then stream.c & dcc-send.c adjust the fps to tweak the stream's 
      bitrate. This is what we want.

 * use_video_encoder_rate_control == false &&
   is_low_bandwidth == false
   -> Then there is no bitrate control whatsoever. I'll talk more about 
      this case in the next section.

So to test how the generic bitrate control algorithm behaves and 
establish a baseline I recommend to:

 * Apply the attached patch. The first hunk will let you track the 
   dcc-send's frame drops, while the second will let you force 
   is_low_bandwidth to true simply by setting SPICE_CAP_BR=1.

 * And set SPICE_DISABLE_ADAPTIVE_STREAMING=1 when running spicy.

To make sure it worked check that you have no line containing 
"stream_report" in the server log and check for lines containing 
"generic frame drop".

It can be useful to tail your server log through:

  egrep "(handle_pong|reattach_stream|marshall_stream|set_bit_rate|: jpeg)"

Testing this you'll notice that reducing the available bandwidth results 
in freezes and recovery times that take a dozen second or more. This is 
because it's only after a sufficient backlog has been created (possibly 
a single large video frame in the queue) that the server drops a frame 
which makes the bitrate control aware that something is wrong. This 
issue is common to all algorithms when all they have to rely on is 
server frame drop notifications.

Here are the flaws I see in this generic bitrate control algorithm:
 * The bitrate is adjusted only once per second. So you get drops for 1 
   second before things even have a chance of getting better.

 * Even after being notified of a server frame drop the generic code 
   continues trying to push as much data as before for up to a second.
   But server frame drops indicate the server queue is full and 
   delaying action will delay a return to normal.

 * If you halve the network bandwidth it will take 15 seconds before it 
   halves the framerate from 30 fps down to 15 (one adjustement per 
   second). With the 15 seconds delay the network queue is sure to still 
   be full so it overshoots and drops the framerate down to 1 fps before 
   increasing it again.

 * It never stabilizes the bitrate. So as soon as there are no drops 
   for 1 second it increases the framerate until there are drops again, 
   then it decreases the framerate until there are no drops. Rince, 
   repeat. The result is you never have smooth video.

3. Testing the no-bitrate-control-whatsoever case
-------------------------------------------------

To test this mode, don't set SPICE_CAP_BR=1 when starting the server, 
and start the client with SPICE_DISABLE_ADAPTIVE_STREAMING=1.

Then verify in your server log that the initial bitrate estimate is 
10Mbps or greater (which is very likely, I have seen values as high as 
73Gbps).

What happens then is that in before_reattach_stream() agent->drops is 
never updated in the first loop because is_low_bandwidth == false. Then 
the second loop never updates agent->fps since drops == 0.

So how can this works?

In the logs this mode results in regular 'generic frame drop' messages 
despite agent->fps being set to 30. This seems to imply frames get 
bunched up. But there are in fact way more dropped frames than these 
messages imply.

My theory is that with the network queue full the server stays blocked 
longer trying to send the frames, causing it to ignore most frames drawn 
by the application (or maybe even preventing the application from 
drawing them).

But having the server block on network traffic does not seem right.

4. Testing the video encoders with server-drops-only
----------------------------------------------------

To test this configuration, simply apply the patch on top of the 
previous one and test as you did for case 2.

In this mode the bitrate control is handled by the video encoder. With 
the initial bitrate estimate being too high it will first spend some 
time dropping the bitrate before stabilizing on a value suitably low.

Unlike in the previous cases the image quality will be lowered to 
preserve a reasonable framerate.

-- 
Francois Gouget <fgouget at codeweavers.com>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: generic-brc.diff
Type: text/x-diff
Size: 1584 bytes
Desc: generic-brc.diff
URL: <https://lists.freedesktop.org/archives/spice-devel/attachments/20161027/cbfdda82/attachment.diff>