[Bug 785745] New: [gl] glcolorscale passthrough mode causing playback failure on video resolution change

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=785745

            Bug ID: 785745
           Summary: [gl] glcolorscale passthrough mode causing playback
                    failure on video resolution change
    Classification: Platform
           Product: GStreamer
           Version: git master
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: Normal
         Component: gst-plugins-bad
          Assignee: gstreamer-bugs at lists.freedesktop.org
          Reporter: samuelh at rd.bbc.co.uk
        QA Contact: gstreamer-bugs at lists.freedesktop.org
     GNOME version: ---

Created attachment 356793
  --> https://bugzilla.gnome.org/attachment.cgi?id=356793&action=edit
Excerpt of log for the issue described

I've been using GStreamer on a Raspberry Pi 2 to display fullscreen DASH. There
is no X session involved, it's using OpenGL ES directly on the framebuffer. As
gst-launch-1.0 starts with a window size equal to the video resolution of the
first DASH representation received and not fullscreen, I use a custom pipeline
to resize the video window to match the output screen size. My pipeline is
below:

gst-launch-1.0 souphttpsrc location=$MPD_URL ! dashdemux name=d \
    d. ! video/quicktime ! queue ! qtdemux ! h264parse ! omxh264dec ! queue \
        ! glupload ! glcolorconvert ! glcolorscale \
        ! "video/x-raw(memory:GLMemory),width=$X_RES,height=$Y_RES" ! queue \
        ! glimagesink
    d. ! audio/x-m4a ! queue ! qtdemux ! aacparse ! avdec_aac ! audioconvert \
        ! omxhdmiaudiosink

Where $MPD_URL is the MPD URL of a given stream, and $X_RES and $Y_RES
representing the screen size to display on.

This works fine, unless the MPD contains a video representation with the same
resolution as the screen output. Then, GStreamer has issues when the resolution
changes away from that specified in the capsfilter downstream of the
glcolorscale element.

For example:

* Player starts, output resolution is 1280x720
* dashdemux selects first representation of a much smaller video resolution
than the output resolution
* Playback begins
* Good network conditions mean dashdemux picks a representation with video
resolution equal to that of the output resolution of 1280x720
* Playback continues as normal
* Network conditions dictate that dashdemux changes representation again to a
representation with a different video resolution to the output resolution
* The pipeline ends with a "not-negotiated" error

I've attached an excerpt of a debug log made with
GST_DEBUG="*:3,glfilter:9,basetransform:9,GST_PADS:6" which shows the error
(glcolorscale-passthrough.log.xz). Of particular note seem to be the lines
which state:

0:00:19.704991143   317   0x25d830 WARN           basetransform
gstbasetransform.c:1346:gst_base_transform_setcaps:<gluploadelement0> transform
could not transform video/x-raw(memory:GLMemory), format=(string)RGBA,
width=(int)960, height=(int)540, interlace-mode=(string)progressive,
pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB,
framerate=(fraction)50/1 in anything we support

After a bit of investigation, I discovered that it seemed to be that the
element was stuck in passthrough mode, and that this was causing the playback
to fail because once in passthrough mode the filter could not be brought out of
it should the caps change upstream of the element. By removing the check for
gst_base_transform_is_passthrough() in gst_gl_filter_transform_caps() in
gst-plugins-bad/gst-libs/gst/gl/gstglfilter.c, it now forces it to do a full
evaluation of the caps when it changes, instead of just taking a reference to
the sink pad caps as they arrive.

By removing the aforementioned check for ::is_passthrough(), this has fixed the
issue for me. I will also attach a patch that I'm now using internally. This
has fixed the issue for me, but as I'm not an expert in either OpenGL nor the
GstBaseTransform side of things I would like to ask what the GStreamer
community's take on this is. I can't find any other GstBaseTransform element
doing what GstGlFilter is doing with that ::is_passthrough() call, so I thought
it a tad suspect.

If you need a test stream, I have created a standalone file which replicates
the behaviour of dashdemux which I can provide. Unfortunately, the video is too
large to upload here (9.1MB and the file size limit is 3.6MB), but I'm looking
into other ways of getting it to you.

For reference, this is running from Git Master as of about three weeks ago with
a few patches pulled in for gl and omx improvements ontop of that, and was also
reproducible on the 1.12.1 release.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.