Improve performance on screen capture using CUDA.

[Seungha Yang]

Hi Davide,

I wonder why you need "memory:SystemMemory" and d3d11download there, instead of "memory:D3D11Memory" 
in first place.
d3d11convert with autovideosink (d3d11videosink by default) will not require any redundant memory copy
operations (which is processing bottleneck in various cases) 

By the way, one requirement for GstD3D11 <-> GstCuda interop is that 
both D3D11 and CUDA objects must belong to the same GPU 
(it still requires GPU to GPU copy with D3D11/CUDA interop, and it's not near zero-cost operation).
Otherwise (d3d11screencapturesrc and CUDA use different GPU), cudaupload/cudadownload elements will do super inefficient memory copy operations, 
Specifically, copying D3D11 GPU memory into system memory, then copy the system memory into CUDA memory.

In short, it depends on your use case, but I expect CUDA may not show significant performance improvement.

> no element "cudaupload"

Possible reasons:
- Your software is running on non-NVIDIA GPU environment
- Registry file needs to be deleted and updated again
- Your NVIDIA GPU does not support encoder and decoder 
  (this should be fixed IMHO, CUDA convert/upload/download don't require en/decoders)

Regards,
Seungha

-----Original Message-----
From: gstreamer-devel <gstreamer-devel-bounces at lists.freedesktop.org> On Behalf Of Davide Perini via gstreamer-devel
Sent: Sunday, February 5, 2023 9:20 PM
To: gstreamer-devel at lists.freedesktop.org
Cc: Davide Perini <perini.davide at dpsoftware.org>
Subject: Improve performance on screen capture using CUDA.

Hi all,
I'm recording the screen using a pipeline like this:

// ./gst-launch-1.0 d3d11screencapturesrc monitor-handle=221948 ! 
d3d11convert ! d3d11download ! autovideosink using this pad:
video/x-raw(memory:SystemMemory),width=INTERNAL_SCALING_X,height=INTERNAL_SCALING_Y,sync=false,

I have seen that in GStreamer 1.22 there are some features that uses Nvidia CUDA, can I optimize my pipeline using CUDA?

// ./gst-launch-1.0 d3d11screencapturesrc ! cudaconvertscale ! 
cudadownload ! autovideosink
using this pad:
video/x-raw(memory:CUDAMemory),width=INTERNAL_SCALING_X,height=INTERNAL_SCALING_Y,sync=false,

Does CUDA help in improving performance in this use case?

In any case I cannot use CUDA since I got this error:

WARNING: erroneous pipeline: no element "cudaupload"

Any idea?

Thanks
Davide