Media Foundation NVIDIA H.264 Encoder

[Ilya Aleshkov]

Hi,
I'd like to use up to 8 mfh264enc elements at the same time.
I feed them manually (appsrc) from Basler cameras.
Everything works great with Intel UHD Graphics 620 (on my Surface Pro 6).
It's really great to have device-agnostic, hardware-accelerated media
processing on Windows. Thanks to Media Foundation Transform API and
GStreamer 1.18.x
But I've faced a real problem with my Nvidia graphics card (GTX 1050 Ti):
The first 2-3 times after reboot everything works without any issues.
After that I get errors like this:

0:00:02.775064000  9884 000001E4C5194380 WARN             mftransform
gstmftransform.cpp:967:gst_mf_transform_set_output_type: MediaFoundation
call failed: 0xc00d6d76, The input type is not supported for D3D device.
0:00:03.361119000  9884 000001E4C5194380 ERROR             mfvideoenc
gstmfvideoenc.cpp:194:gst_mf_video_enc_set_format:<mfh264enc0> Couldn't set
output type
0:00:04.631488000  9884 000001E4C5194380 WARN            videoencoder
gstvideoencoder.c:755:gst_video_encoder_setcaps:<mfh264enc0> rejected caps
video/x-raw, format=(string)NV12, width=(int)1200, height=(int)1000,
framerate=(fraction)80/1, multiview-mode=(string)mono,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive
0:00:05.328089000  9884 000001E4C5194380 WARN             mftransform
gstmftransform.cpp:967:gst_mf_transform_set_output_type: MediaFoundation
call failed: 0xc00d6d76, The input type is not supported for D3D device.
0:00:05.342697000  9884 000001E4C5194380 ERROR             mfvideoenc
gstmfvideoenc.cpp:194:gst_mf_video_enc_set_format:<mfh264enc0> Couldn't set
output type
0:00:05.352995000  9884 000001E4C5194380 WARN            videoencoder
gstvideoencoder.c:755:gst_video_encoder_setcaps:<mfh264enc0> rejected caps
video/x-raw, format=(string)NV12, width=(int)1200, height=(int)1000,
framerate=(fraction)80/1, multiview-mode=(string)mono,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive
0:00:05.372768000  9884 000001E4C5194380 WARN                GST_PADS
gstpad.c:4303:gst_pad_peer_query:<capsfilter0:src> could not send sticky
events
0:00:05.413309000  9884 000001E4C5194380 WARN             mftransform
gstmftransform.cpp:967:gst_mf_transform_set_output_type: MediaFoundation
call failed: 0xc00d6d76, The input type is not supported for D3D device.
0:00:05.657361000  9884 000001E4C5194380 ERROR             mfvideoenc
gstmfvideoenc.cpp:194:gst_mf_video_enc_set_format:<mfh264enc0> Couldn't set
output type
0:00:05.768150000  9884 000001E4C5194380 WARN            videoencoder
gstvideoencoder.c:755:gst_video_encoder_setcaps:<mfh264enc0> rejected caps
video/x-raw, format=(string)NV12, width=(int)1200, height=(int)1000,
framerate=(fraction)80/1, multiview-mode=(string)mono,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive
0:00:06.439141000  9884 000001E4C5194380 WARN             mftransform
gstmftransform.cpp:967:gst_mf_transform_set_output_type: MediaFoundation
call failed: 0xc00d6d76, The input type is not supported for D3D device.
0:00:06.535926000  9884 000001E4C5194380 ERROR             mfvideoenc
gstmfvideoenc.cpp:194:gst_mf_video_enc_set_format:<mfh264enc0> Couldn't set
output type
0:00:06.773513000  9884 000001E4C5194380 WARN            videoencoder
gstvideoencoder.c:755:gst_video_encoder_setcaps:<mfh264enc0> rejected caps
video/x-raw, format=(string)NV12, width=(int)1200, height=(int)1000,
framerate=(fraction)80/1, multiview-mode=(string)mono,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive
0:00:07.577236000  9884 000001E4C5194380 WARN                 basesrc
gstbasesrc.c:3127:gst_base_src_loop:<videotestsrc0> error: Internal data
stream error.
0:00:07.620275000  9884 000001E4C5194380 WARN                 basesrc
gstbasesrc.c:3127:gst_base_src_loop:<videotestsrc0> error: streaming
stopped, reason not-negotiated (-4)
ERROR: from element /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0:
Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop ():
/GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...

As a result, I get 1-3 empty files (of 8) every run.
The easiest way to reproduce this issue it to launch this cmd file:

start videotestsrc_mfh264enc.bat testsrc_1.mkv
start videotestsrc_mfh264enc.bat testsrc_2.mkv
start videotestsrc_mfh264enc.bat testsrc_3.mkv
start videotestsrc_mfh264enc.bat testsrc_4.mkv
start videotestsrc_mfh264enc.bat testsrc_5.mkv
start videotestsrc_mfh264enc.bat testsrc_6.mkv
start videotestsrc_mfh264enc.bat testsrc_7.mkv
start videotestsrc_mfh264enc.bat testsrc_8.mkv

As you can see it uses another cmd file as a subroutine
(videotestsrc_mfh264enc.bat):

gst-launch-1.0 -e videotestsrc num-buffers=800 pattern=ball !
video/x-raw,format=NV12,width=1200,height=1000,framerate=80/1 ! mfh264enc !
h264parse ! matroskamux ! filesink location=%~n1.mkv

I've tested it with mfh265enc as well - the same issue.
What do you think about it?
Now I'm going to dive into MFTs API and  GstMFTransform sources.

Thanks,
Ilya
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20201106/d6cca904/attachment-0001.htm>