[Bug 663248] New: colorspace: bad memory accesses in orc code for odd width buffers

[GStreamer (bugzilla.gnome.org)]

https://bugzilla.gnome.org/show_bug.cgi?id=663248
  GStreamer | gst-plugins-bad | git

           Summary: colorspace: bad memory accesses in orc code for odd
                    width buffers
    Classification: Platform
           Product: GStreamer
           Version: git
        OS/Version: Linux
            Status: NEW
          Severity: major
          Priority: Normal
         Component: gst-plugins-bad
        AssignedTo: gstreamer-bugs at lists.freedesktop.org
        ReportedBy: vincent.penquerch at collabora.co.uk
         QAContact: gstreamer-bugs at lists.freedesktop.org
     GNOME version: ---


I've tracked them down to the ORC versions of some of the AYUV related routines
when run on buffers with odd width. Since AYUV is not subsampled, I guess the
issue is with the other format converted from/to.

I'm not 100% sure, but I think these AYUV related routines process two pixels
at a time, and the caller will pass (width+1)/2. There seems to be no way to
convert an extra column alone, save custom code each time.

Someone who is fluent with ORC code could maybe look at
cogorc_convert_UYVY_AYUV and say if I'm correct about that two pixels thing. We
might then need to either convert it to single pixel at a time, or have an
alternate version to use for odd widths.

Easily checked by running the pipelines/colorspace test. The crash goes with
this patch (which removes the tests for these AYUV related formats (though I've
not run that with valgrind, so it might be there still are more bad accesses):


diff --git a/gst/colorspace/colorspace.c b/gst/colorspace/colorspace.c
index bb8c58d..d3d804d 100644
--- a/gst/colorspace/colorspace.c
+++ b/gst/colorspace/colorspace.c
@@ -2397,7 +2397,7 @@ static const ColorspaceTransform transforms[] = {
       COLOR_SPEC_NONE, TRUE, convert_UYVY_I420},
   {GST_VIDEO_FORMAT_UYVY, COLOR_SPEC_NONE, GST_VIDEO_FORMAT_YUY2,
       COLOR_SPEC_NONE, TRUE, convert_UYVY_YUY2},
-  {GST_VIDEO_FORMAT_UYVY, COLOR_SPEC_NONE, GST_VIDEO_FORMAT_AYUV,
+  {GST_VIDEO_FORMAT_UYVY+88888, COLOR_SPEC_NONE, GST_VIDEO_FORMAT_AYUV,
       COLOR_SPEC_NONE, TRUE, convert_UYVY_AYUV},
   {GST_VIDEO_FORMAT_UYVY, COLOR_SPEC_NONE, GST_VIDEO_FORMAT_Y42B,
       COLOR_SPEC_NONE, TRUE, convert_UYVY_Y42B},
@@ -2406,11 +2406,11 @@ static const ColorspaceTransform transforms[] = {

   {GST_VIDEO_FORMAT_AYUV, COLOR_SPEC_NONE, GST_VIDEO_FORMAT_I420,
       COLOR_SPEC_NONE, TRUE, convert_AYUV_I420},
-  {GST_VIDEO_FORMAT_AYUV, COLOR_SPEC_NONE, GST_VIDEO_FORMAT_YUY2,
+  {GST_VIDEO_FORMAT_AYUV+88888, COLOR_SPEC_NONE, GST_VIDEO_FORMAT_YUY2,
       COLOR_SPEC_NONE, TRUE, convert_AYUV_YUY2},
-  {GST_VIDEO_FORMAT_AYUV, COLOR_SPEC_NONE, GST_VIDEO_FORMAT_UYVY,
+  {GST_VIDEO_FORMAT_AYUV+88888, COLOR_SPEC_NONE, GST_VIDEO_FORMAT_UYVY,
       COLOR_SPEC_NONE, TRUE, convert_AYUV_UYVY},
-  {GST_VIDEO_FORMAT_AYUV, COLOR_SPEC_NONE, GST_VIDEO_FORMAT_Y42B,
+  {GST_VIDEO_FORMAT_AYUV+8888, COLOR_SPEC_NONE, GST_VIDEO_FORMAT_Y42B,
       COLOR_SPEC_NONE, TRUE, convert_AYUV_Y42B},
   {GST_VIDEO_FORMAT_AYUV, COLOR_SPEC_NONE, GST_VIDEO_FORMAT_Y444,
       COLOR_SPEC_NONE, TRUE, convert_AYUV_Y444},
@@ -2421,7 +2421,7 @@ static const ColorspaceTransform transforms[] = {
       COLOR_SPEC_NONE, TRUE, convert_Y42B_YUY2},
   {GST_VIDEO_FORMAT_Y42B, COLOR_SPEC_NONE, GST_VIDEO_FORMAT_UYVY,
       COLOR_SPEC_NONE, TRUE, convert_Y42B_UYVY},
-  {GST_VIDEO_FORMAT_Y42B, COLOR_SPEC_NONE, GST_VIDEO_FORMAT_AYUV,
+  {GST_VIDEO_FORMAT_Y42B+8888, COLOR_SPEC_NONE, GST_VIDEO_FORMAT_AYUV,
       COLOR_SPEC_NONE, TRUE, convert_Y42B_AYUV},
   {GST_VIDEO_FORMAT_Y42B, COLOR_SPEC_NONE, GST_VIDEO_FORMAT_Y444,
       COLOR_SPEC_NONE, TRUE, convert_Y42B_Y444},

-- 
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.