I need help verifying there's a bug in the Intel Media SDK Gstreamer element

[Ben Rush]

Doh! I noticed a typo in my email. The second gst-launch script should be:

 *gst-launch-1.0.exe rtspsrc location=rtsp://address/live ! decodebin !
'video/x-raw(memory:MFXSurface),format=BGRA' ! autovideosink*

My bad.

On Tue, Jun 18, 2019 at 12:37 PM Ben Rush <ben at ben-rush.net> wrote:

> The whole point of this email is to verify I've encountered a bug, and not
> that I'm doing some stupid.
>
> I've been successfully using the Intel Media SDK GStreamer element from
> their repo here:
> https://github.com/intel/gstreamer-media-SDK/tree/topic_linux_and_window for
> a while now. My successful setup is I've got a camera connected to a
> machine (a Windows machine), and then I'm using the GStreamer's RTSP server
> to take the camera feed, encode it on the GPU (with the help of the Intel
> GStreamer h265/h264 plugins), and then broadcast it. There were a few bumps
> along the way (mostly bugs and issues in their code), but I've since got it
> resolved and have things working. GStreamer is awesome. My point thus far
> is I've been able to get these Intel plugins to work with the rest of
> GStreamer.
>
> I've been given another use case, and I'm encountering what appears to be
> another issue in Intel's code. I would normally think I'm doing something
> dumb, but given the # of issues I ran into when getting their elements to
> work in the aforementioned setup, I'm considering I've run into another
> bump.
>
> Essentially I'm NOW writing a video player, and need to decode traffic. So
> I'm on the player side of the equation now. Again, I'm using the Intel
> Media SDK to decode. Now, according to gst-inspect, the output pins for
> either the h264 or h265 intel decoders should produce BGRA. This is
> important because the user interface control I'm using doesn't support
> NV12, and requires a BGRA buffer. I don't want to do the conversion from
> NV12 to RGB* on the CPU 'cause that defeats the whole reason I'm using the
> Intel Media SDK in the first place. Example output from gst-inspect on
> their h265 element:
>
> Pad Templates:
>   SINK template: 'sink'
>     Availability: Always
>     Capabilities:
>       video/x-h265
>               alignment: au
>                 profile: { (string)main, (string)main-10 }
>           stream-format: byte-stream
>
>   SRC template: 'src'
>     Availability: Always
>     Capabilities:
>       video/x-raw(memory:MFXSurface)
>                  format: { (string)NV12, (string)BGRA, (string)P010_10LE,
> (string)YUY2, (string)ENCODED }
>                   width: [ 1, 2147483647 ]
>                  height: [ 1, 2147483647 ]
>               framerate: [ 0/1, 2147483647/1 ]
>       video/x-raw
>                  format: { (string)NV12, (string)BGRA, (string)P010_10LE,
> (string)YUY2 }
>                   width: [ 1, 2147483647 ]
>                  height: [ 1, 2147483647 ]
>               framerate: [ 0/1, 2147483647/1
> However, NOTHING I do successfully decodes into BGRA. I can write the
> following gst-launch script and see the video stream just fine when
> specifying NV12:
>
> *gst-launch-1.0.exe rtspsrc location=rtsp://address/live ! decodebin !
> 'video/x-raw(memory:MFXSurface),format=NV12' ! autovideosink*
>
> If I change the format to BGRA, however, the player doesn't pop up:
>
>  *gst-launch-1.0.exe rtspsrc location=rtsp://address/live ! decodebin !
> 'video/x-raw(memory:MFXSurface),format=NV12' ! autovideosink*
>
> What's weird is I see both the CPU and GPU chewing away doing the above
> script, but nothing displays. By dumping out the dot files, I see the
> pipeline fails to successfully enter the PLAY state. Digging further...
>
> If I attach and debug the Intel MFX element and run the above script that
> fails, I see the following line fails:
> https://github.com/intel/gstreamer-media-SDK/blob/94ab6055b60e9d67a3ccb6b13fddb6c3076e204d/gst/mfx/gstmfxdec.c#L424.
> It looks like this:
>
> return gst_video_decoder_finish_frame (vdec, frame);
>
> The function returns GST_FLOW_NOT_NEGOTIATED.  Setting GST_DEBUG=4, I see
> the following warning which further indicates that some kind of negotiation
> failed:
>
> WARN                GST_PADS
> gstpad.c:4230:gst_pad_peer_query:<mfxdec_h264-0:src> could not send sticky
> events
>
> What I need is confirmation that the above gst-launch commands OUGHT to
> give me BGRA at the autovideosink if things were working correctly in the
> Intel decoders. Everything thus far seems to indicate I'm running into some
> bug. To be doubly sure, I see the same behavior when I replace
> autovideosink with appsink and write some C++ around it.
>
> Thanks for your expertise and extra pair of eyes.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20190618/b8793987/attachment-0001.html>