[Libva] [PATCH 1/3] jpeg_enc: Avoid integer overflow while doing quality factor scaling

[Muppavarapu, Sirisha]

+1 LGTM

-----Original Message-----
From: Libva [mailto:libva-bounces at lists.freedesktop.org] On Behalf Of sreerenj.balachandran at intel.com
Sent: Thursday, March 05, 2015 3:04 AM
To: libva at lists.freedesktop.org
Subject: [Libva] [PATCH 1/3] jpeg_enc: Avoid integer overflow while doing quality factor scaling

From: Sreerenj Balachandran <sreerenj.balachandran at intel.com>

For eg: The uint8_t will simple overflow if submitted quality factor is 1 (5000/1).
---
 src/gen8_mfc.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/src/gen8_mfc.c b/src/gen8_mfc.c index 314b882..698dcae 100644
--- a/src/gen8_mfc.c
+++ b/src/gen8_mfc.c
@@ -2674,7 +2674,7 @@ gen8_mfc_jpeg_fqm_state(VADriverContextP ctx,
                         struct intel_encoder_context *encoder_context,
                         struct encode_state *encode_state)  {
-    uint8_t quality = 0;
+    unsigned int quality = 0;
     uint32_t temp, i = 0, j = 0, dword_qm[32];
     VAEncPictureParameterBufferJPEG *pic_param;
     VAQMatrixBufferJPEG *qmatrix;
@@ -2706,7 +2706,9 @@ gen8_mfc_jpeg_fqm_state(VADriverContextP ctx,
         qmatrix->load_lum_quantiser_matrix = 1;
         qmatrix->load_chroma_quantiser_matrix = (pic_param->num_components > 1) ? 1 : 0;
     }   
- 
+
+    if (quality > 100)
+      quality = 100;
     quality = (quality < 50) ? (5000/quality) : (200 - (quality*2)); 
     quality = (quality == 0) ? 1 : quality;
     
--
1.9.1

_______________________________________________
Libva mailing list
Libva at lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/libva