[Bug 767712] xvimage: Missing sanity checking for allocation sizes for various video formats

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=767712

--- Comment #28 from Duncan Palmer <dunk.palmer at gmail.com> ---
Thanks Sebastian,

I'll update the patch with your suggestions this week.

(In reply to Sebastian Dröge (slomo) from comment #27)
> 
> @@ +438,3 @@
> +    } else if (GST_VIDEO_FORMAT_INFO_IS_RGB (info)) {
> +      expected_size = padded_height * padded_width *
> +          GST_VIDEO_FORMAT_INFO_PSTRIDE (info, 0);
> 
> My guess is that this should be "padded_height * GST_ROUND_UP_4(padded_width
> * pstride)"

I'm not sure if your suggestion is correct e.g. if we're using
GST_VIDEO_FORMAT_RGB, where pstride[0] == 3. In this case, I would have though
that the stride should be width * 3. Have I got this wrong?

Dunk

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.