[Spice-devel] [spice-server v3 3/3] clang: fix warning and possible wrong computation

[Victor Toso]

From: Victor Toso <me at victortoso.com>

Clang's warning about absolute value below is not entirely correct as
the difference between the two bitrate values could be negative but
using abs() could be wrong.

> gstreamer-encoder.c:513:17: error: taking the absolute value of unsigned
> type 'unsigned long' has no effect
>     } else  if (abs(bit_rate - encoder->video_bit_rate) > encoder->video_bit_rate * SPICE_GST_VIDEO_BITRATE_MARGIN) {
>                 ^
> gstreamer-encoder.c:513:17: note: remove the call to 'abs' since unsigned
> values cannot be negative
> } else if (abs(bit_rate - encoder->video_bit_rate) > encoder->video_bit_rate * SPICE_GST_VIDEO_BITRATE_MARGIN) {
>            ^~~

Signed-off-by: Victor Toso <victortoso at redhat.com>
---
 server/gstreamer-encoder.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/server/gstreamer-encoder.c b/server/gstreamer-encoder.c
index 04f0c02f..ac38f7da 100644
--- a/server/gstreamer-encoder.c
+++ b/server/gstreamer-encoder.c
@@ -505,12 +505,19 @@ static void set_gstenc_bitrate(SpiceGstEncoder *encoder);
 
 static void set_video_bit_rate(SpiceGstEncoder *encoder, uint64_t bit_rate)
 {
+    uint64_t diff;
+    gdouble margin;
+
     if (encoder->video_bit_rate != bit_rate &&
         encoder->gstenc_bitrate_is_dynamic) {
         encoder->video_bit_rate = bit_rate;
         set_gstenc_bitrate(encoder);
+        return;
+    }
 
-    } else  if (abs(bit_rate - encoder->video_bit_rate) > encoder->video_bit_rate * SPICE_GST_VIDEO_BITRATE_MARGIN) {
+    diff = labs((int64_t) bit_rate - (int64_t) encoder->video_bit_rate);
+    margin = encoder->video_bit_rate * SPICE_GST_VIDEO_BITRATE_MARGIN;
+    if (diff > margin) {
         encoder->video_bit_rate = bit_rate;
         set_pipeline_changes(encoder, SPICE_GST_VIDEO_PIPELINE_BITRATE);
     }
-- 
2.15.1