[Bug 762856] New: It is possible hit Buffer Overflow - Array Index Out of Bounds in fill_iq_matrix_8x8 for H264

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=762856

            Bug ID: 762856
           Summary: It is possible hit Buffer Overflow - Array Index Out
                    of Bounds in fill_iq_matrix_8x8 for H264
    Classification: Platform
           Product: GStreamer
           Version: git master
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: Normal
         Component: gstreamer-vaapi
          Assignee: gstreamer-bugs at lists.freedesktop.org
          Reporter: siew.hoon.lim at intel.com
        QA Contact: gstreamer-bugs at lists.freedesktop.org
                CC: bsreerenj at gmail.com, vjaquez at igalia.com
     GNOME version: ---

I still got 1 KW issue that I'm not able to confirm is it possible an issue. 
I'm look on avdec_h264 side how they handle it and also check thru libva and
Intel VA driver side.
It look like really an issue will pretty sure will hit the buffer overflow -
Array Index out of bounds n=6 and chroma_format_idc = 3, when the for loop i=
2...5. 
Please advise will it be an issue?


>From libva
/** H.264 Inverse Quantization Matrix Buffer */
typedef struct _VAIQMatrixBufferH264    
{        
    /** \brief 4x4 scaling list, in raster scan order. */    
    unsigned char ScalingList4x4[6][16];    
    /** \brief 8x8 scaling list, in raster scan order. */    
    unsigned char ScalingList8x8[2][64];    
} VAIQMatrixBufferH264;

>From gsth264parser.h
struct _GstH264PPS
{
  guint8 transform_8x8_mode_flag;        
  guint8 scaling_lists_4x4[6][16];    
  guint8 scaling_lists_8x8[6][64];    
...
};


static void
fill_iq_matrix_8x8 (VAIQMatrixBufferH264 * iq_matrix, const GstH264PPS * pps,
    const GstH264SPS * sps)
{
  guint i, n;

  /* If chroma_format_idc != 3, there are up to 2 8x8 scaling lists */
  if (!pps->transform_8x8_mode_flag)
    return;

  g_assert (G_N_ELEMENTS (iq_matrix->ScalingList8x8) >= 2);
  g_assert (G_N_ELEMENTS (iq_matrix->ScalingList8x8[0]) == 64);

  n = (sps->chroma_format_idc != 3) ? 2 : 6;
  for (i = 0; i < n; i++) {
    gst_h264_quant_matrix_8x8_get_raster_from_zigzag (iq_matrix->ScalingList8x8
        [i], pps->scaling_lists_8x8[i]);
  }
}

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.