[Bug 777506] New: vaapidecoder_h264: Recursive loop and segfault after a few seconds

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=777506

            Bug ID: 777506
           Summary: vaapidecoder_h264: Recursive loop and segfault after a
                    few seconds
    Classification: Platform
           Product: GStreamer
           Version: 1.10.1
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: Normal
         Component: gstreamer-vaapi
          Assignee: gstreamer-bugs at lists.freedesktop.org
          Reporter: nnoell3 at gmail.com
        QA Contact: gstreamer-bugs at lists.freedesktop.org
                CC: bsreerenj at gmail.com, vjaquez at igalia.com
     GNOME version: ---

Created attachment 343830
  --> https://bugzilla.gnome.org/attachment.cgi?id=343830&action=edit
Backtrace

Hi,

We just realized that the vaapi h264 decoder cannot decode a specific rtsp
source with the following pipeline:

gst-launch-1.0 rtspsrc location=rtsp://... ! rtph264depay ! h264parse !
 vaapidecodebin ! fakesink

It segfaults after a few seconds. The backtrace shows many
*gst_vaapi_picture_destroy* being called recursively tons of times and the
segfault happens in glib (probably because of OOM). I have attached a reduced
version of the backtrace showing the beginning and end of it.

Also, adding a caps filter between rtph264depay and h264parse to change the 
alignment from *nal* to *au* seems to solve the problem:

gst-launch-1.0 rtspsrc ! rtph264depay !
video/x-h264,stream-format=byte-stream,alignment=au ! h264parse ! vaapidecode !
autovideosink

It would be great for the decoder to decode any kind of alignment without
applying a capsfilter.

Please, let us know if you need more information. Thanks.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.