[Bug 734156] New: androidmedia doesn't calculate framesize for COLOR_FormatYUV420Planar correctly

[GStreamer (bugzilla.gnome.org)]

https://bugzilla.gnome.org/show_bug.cgi?id=734156
  GStreamer | gst-plugins-bad | 1.4.0

           Summary: androidmedia doesn't calculate framesize for
                    COLOR_FormatYUV420Planar correctly
    Classification: Platform
           Product: GStreamer
           Version: 1.4.0
        OS/Version: Windows
            Status: UNCONFIRMED
          Severity: major
          Priority: Normal
         Component: gst-plugins-bad
        AssignedTo: gstreamer-bugs at lists.freedesktop.org
        ReportedBy: howard at rti.com
         QAContact: gstreamer-bugs at lists.freedesktop.org
     GNOME version: ---


In sys/androidmedia/gstamc.c, line 2479


Have this:

    case COLOR_FormatYUV420Planar:{
      if (stride == 0 || slice_height == 0) {
        GST_ERROR ("Stride or slice height is 0");
        return FALSE;
      }

      frame_size =
          stride * slice_height + 2 * (((stride + 1) / 2) * (slice_height +
              1) / 2);
      break;
    }


so for a width(stride)=640, height(slice_height)=480, I think that the
frame_size should be 460800 bytes, but the calculation gives 461120 bytes.

bad calculation is

      frame_size = 640*480 + 2 * (((640+1)/2) * (480+1)/2) = 461120 (use
integer division)

good calculation is

      frame_size = 640*480 + 2 * ((640+1)/2) * ((480+1)/2) = 460800

I think that the code is missing some parens...

      frame_size =
          stride * slice_height + 2 * ( ((stride + 1) / 2) * ((slice_height +
              1) / 2) );

-- 
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.