Improve performance on screen capture using CUDA.

[Davide Perini]

Hi Seungha,
thanks for the answer, I really appreciate it.

I think that part of the problem is that I didn't quite understand how 
this pipeline worked,
now that you explained me it, it's more clear.

I'm capturing the screen in order to process the captured frames, get an 
average color for that frames and send it to a microcontroller,
you know, like an ambilight :)

As you pointed out there is no reason to use "memory:SystemMemory" and 
d3d11download.

I switched to "memory:D3D11Memory" and removed d3d11download and it 
works well,
thanks for the suggestion and for the great work here on GStreamer Seungha.

Kind Regards,
Davide


Il 05/02/2023 21:29, Seungha Yang ha scritto:
> Hi Davide,
>
> I wonder why you need "memory:SystemMemory" and d3d11download there, instead of "memory:D3D11Memory"
> in first place.
> d3d11convert with autovideosink (d3d11videosink by default) will not require any redundant memory copy
> operations (which is processing bottleneck in various cases)
>
> By the way, one requirement for GstD3D11 <-> GstCuda interop is that
> both D3D11 and CUDA objects must belong to the same GPU
> (it still requires GPU to GPU copy with D3D11/CUDA interop, and it's not near zero-cost operation).
> Otherwise (d3d11screencapturesrc and CUDA use different GPU), cudaupload/cudadownload elements will do super inefficient memory copy operations,
> Specifically, copying D3D11 GPU memory into system memory, then copy the system memory into CUDA memory.
>
> In short, it depends on your use case, but I expect CUDA may not show significant performance improvement.
>
>> no element "cudaupload"
> Possible reasons:
> - Your software is running on non-NVIDIA GPU environment
> - Registry file needs to be deleted and updated again
> - Your NVIDIA GPU does not support encoder and decoder
>    (this should be fixed IMHO, CUDA convert/upload/download don't require en/decoders)
>
> Regards,
> Seungha
>
> -----Original Message-----
> From: gstreamer-devel<gstreamer-devel-bounces at lists.freedesktop.org>  On Behalf Of Davide Perini via gstreamer-devel
> Sent: Sunday, February 5, 2023 9:20 PM
> To:gstreamer-devel at lists.freedesktop.org
> Cc: Davide Perini<perini.davide at dpsoftware.org>
> Subject: Improve performance on screen capture using CUDA.
>
> Hi all,
> I'm recording the screen using a pipeline like this:
>
> // ./gst-launch-1.0 d3d11screencapturesrc monitor-handle=221948 !
> d3d11convert ! d3d11download ! autovideosink using this pad:
> video/x-raw(memory:SystemMemory),width=INTERNAL_SCALING_X,height=INTERNAL_SCALING_Y,sync=false,
>
> I have seen that in GStreamer 1.22 there are some features that uses Nvidia CUDA, can I optimize my pipeline using CUDA?
>
> // ./gst-launch-1.0 d3d11screencapturesrc ! cudaconvertscale !
> cudadownload ! autovideosink
> using this pad:
> video/x-raw(memory:CUDAMemory),width=INTERNAL_SCALING_X,height=INTERNAL_SCALING_Y,sync=false,
>
> Does CUDA help in improving performance in this use case?
>
> In any case I cannot use CUDA since I got this error:
>
> WARNING: erroneous pipeline: no element "cudaupload"
>
> Any idea?
>
> Thanks
> Davide