Failed to create NVENC encoder session, ret=10 (nvh264enc

[Aniket Hande]

Hi,

I'm trying to do video transcoding where my source codec is MPEG-2 and
destination is H264. The overall pipeline is as,
udpsrc -> rtpbin -> depay -> nvdec(GPU0) -> glcolorconvert ->
nvh264enc(GPU1) -> mp4mux

I have two RTX 3060 cards. Since I'm using Gstreamer 1.16 by default all
elements are using GPU0. Two distribute the load as I will be having
multiple parallel transcoding streams I have assigned GPU1 for encoding by
using the "cuda-device-id=1" property of nvh264enc. However I'm facing the
following issue (attached logs). This does not always occur but a few
streams fail. The max number of parallel streams I'm trying is 12. Also I'm
getting a crash in libglib2.0.

Logs:

0:00:00.021675805   222 0x5559e084b920 INFO                   nvenc
gstnvenc.c:377:plugin_init: Created NVEncodeAPI instance, got function table
0:00:00.029537027   222 0x5559e084b920 DEBUG              nvh264enc
gstnvh264enc.c:325:gst_nv_h264_enc_getcaps:<nvh264enc0>   returning caps
video/x-raw, width=(int)[ 16, 4096 ], height=(int)[ 16, 4096 ],
framerate=(fraction)[ 0/1, 2147483647/1 ], format=(string){ NV12, I420 },
interlace-mode=(string){ progressive, mixed, interleaved };
video/x-raw(memory:GLMemory), width=(int)[ 16, 4096 ], height=(int)[ 16,
4096 ], framerate=(fraction)[ 0/1, 2147483647/1 ], format=(string){ NV12,
Y444 }, interlace-mode=(string){ progressive, mixed, interleaved }
0:00:00.031601164   222 0x5559e084b920 DEBUG              nvh264enc
gstnvh264enc.c:325:gst_nv_h264_enc_getcaps:<nvh264enc0>   returning caps
video/x-raw, width=(int)[ 16, 4096 ], height=(int)[ 16, 4096 ],
framerate=(fraction)[ 0/1, 2147483647/1 ], format=(string){ NV12, I420 },
interlace-mode=(string){ progressive, mixed, interleaved };
video/x-raw(memory:GLMemory), width=(int)[ 16, 4096 ], height=(int)[ 16,
4096 ], framerate=(fraction)[ 0/1, 2147483647/1 ], format=(string){ NV12,
Y444 }, interlace-mode=(string){ progressive, mixed, interleaved }
0:00:00.033646178   222 0x5559e084b920 INFO                   nvenc
gstnvenc.c:284:gst_nvenc_create_cuda_context: Initialising CUDA..
0:00:00.056518536   222 0x5559e084b920 INFO                   nvenc
gstnvenc.c:293:gst_nvenc_create_cuda_context: Initialised CUDA
0:00:00.056528395   222 0x5559e084b920 INFO                   nvenc
gstnvenc.c:301:gst_nvenc_create_cuda_context: 2 CUDA device(s) detected
0:00:00.056556831   222 0x5559e084b920 INFO                   nvenc
gstnvenc.c:311:gst_nvenc_create_cuda_context: GPU #0 supports NVENC: yes
(NVIDIA GeForce RTX 3060) (Compute SM 8.6)
0:00:00.056573650   222 0x5559e084b920 INFO                   nvenc
gstnvenc.c:311:gst_nvenc_create_cuda_context: GPU #1 supports NVENC: yes
(NVIDIA GeForce RTX 3060) (Compute SM 8.6)
0:00:00.140664394   222 0x5559e084b920 INFO                   nvenc
gstnvenc.c:333:gst_nvenc_create_cuda_context: Created CUDA context
0x5559e08c1210
0:00:00.144684723   222 0x5559e084b920 ERROR                  nvenc
gstnvbaseenc.c:421:gst_nv_base_enc_open: Failed to create NVENC encoder
session, ret=10
0:00:00.144692262   222 0x5559e084b920 INFO                   nvenc
gstnvenc.c:341:gst_nvenc_destroy_cuda_context: Destroying CUDA context
0x5559e08c1210

dmesg shows something like:
[43700.697479] traps: main[15966] trap int3 ip:7fc0748d2fc1 sp:7fff0fb19ff0
error:0 in libglib-2.0.so.0.5600.4[7fc074881000+114000]
--

*Regards,*
*Aniket*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20220119/8177241a/attachment.htm>