I need help verifying there's a bug in the Intel Media SDK Gstreamer element

[Ben Rush]

The whole point of this email is to verify I've encountered a bug, and not
that I'm doing some stupid.

I've been successfully using the Intel Media SDK GStreamer element from
their repo here:
https://github.com/intel/gstreamer-media-SDK/tree/topic_linux_and_window for
a while now. My successful setup is I've got a camera connected to a
machine (a Windows machine), and then I'm using the GStreamer's RTSP server
to take the camera feed, encode it on the GPU (with the help of the Intel
GStreamer h265/h264 plugins), and then broadcast it. There were a few bumps
along the way (mostly bugs and issues in their code), but I've since got it
resolved and have things working. GStreamer is awesome. My point thus far
is I've been able to get these Intel plugins to work with the rest of
GStreamer.

I've been given another use case, and I'm encountering what appears to be
another issue in Intel's code. I would normally think I'm doing something
dumb, but given the # of issues I ran into when getting their elements to
work in the aforementioned setup, I'm considering I've run into another
bump.

Essentially I'm NOW writing a video player, and need to decode traffic. So
I'm on the player side of the equation now. Again, I'm using the Intel
Media SDK to decode. Now, according to gst-inspect, the output pins for
either the h264 or h265 intel decoders should produce BGRA. This is
important because the user interface control I'm using doesn't support
NV12, and requires a BGRA buffer. I don't want to do the conversion from
NV12 to RGB* on the CPU 'cause that defeats the whole reason I'm using the
Intel Media SDK in the first place. Example output from gst-inspect on
their h265 element:

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-h265
              alignment: au
                profile: { (string)main, (string)main-10 }
          stream-format: byte-stream

  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw(memory:MFXSurface)
                 format: { (string)NV12, (string)BGRA, (string)P010_10LE,
(string)YUY2, (string)ENCODED }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw
                 format: { (string)NV12, (string)BGRA, (string)P010_10LE,
(string)YUY2 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1
However, NOTHING I do successfully decodes into BGRA. I can write the
following gst-launch script and see the video stream just fine when
specifying NV12:

*gst-launch-1.0.exe rtspsrc location=rtsp://address/live ! decodebin !
'video/x-raw(memory:MFXSurface),format=NV12' ! autovideosink*

If I change the format to BGRA, however, the player doesn't pop up:

 *gst-launch-1.0.exe rtspsrc location=rtsp://address/live ! decodebin !
'video/x-raw(memory:MFXSurface),format=NV12' ! autovideosink*

What's weird is I see both the CPU and GPU chewing away doing the above
script, but nothing displays. By dumping out the dot files, I see the
pipeline fails to successfully enter the PLAY state. Digging further...

If I attach and debug the Intel MFX element and run the above script that
fails, I see the following line fails:
https://github.com/intel/gstreamer-media-SDK/blob/94ab6055b60e9d67a3ccb6b13fddb6c3076e204d/gst/mfx/gstmfxdec.c#L424.
It looks like this:

return gst_video_decoder_finish_frame (vdec, frame);

The function returns GST_FLOW_NOT_NEGOTIATED.  Setting GST_DEBUG=4, I see
the following warning which further indicates that some kind of negotiation
failed:

WARN                GST_PADS
gstpad.c:4230:gst_pad_peer_query:<mfxdec_h264-0:src> could not send sticky
events

What I need is confirmation that the above gst-launch commands OUGHT to
give me BGRA at the autovideosink if things were working correctly in the
Intel decoders. Everything thus far seems to indicate I'm running into some
bug. To be doubly sure, I see the same behavior when I replace
autovideosink with appsink and write some C++ around it.

Thanks for your expertise and extra pair of eyes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20190618/31ff22b0/attachment.html>