4K playback optimization help needed

[Nicolas Dufresne]

Le dimanche 26 janvier 2020 à 07:07 -0600, QwjN1Y9mJvamZJ a écrit :
> I would like to ask for some advice regarding how to make 4K playback
> smoother. I don't think the issue is that my PC can't keep up with decoding,
> because if I use /sync = false/ on the video sink, the video plays fast and
> very smooth. Details follow:
> 
> 1. The basic pipeline I tried:
> 
> /gst-launch-1.0.exe filesrc location=4K.mkv ! matroskademux ! h265parse !
> nvh265dec ! glimagesink/
> 
> This works, but the image is a bit "choppy", as in I feel I can see as the
> movie proceeds from one frame onto the next, especially when the camera is
> panning. Initially I assumed that the jumpiness was because of the
> discrepancy between the movie's frame rate (23.967) and that of my monitor
> (60), but the file plays back perfectly smooth with a video player (Windows
> 10 x64, MPC-BE with LAV filters using CUVID hwaccel and EVR renderer). I
> don't think that MPC-BE is using motion interpolation either.
> 
> As to why I'm using these filters: d3d11 filters (d3d11h265dec ->
> glimagesink or d3d11videosink) only produce a white or gray image (perhaps
> because the source is 10bit), avdec_h265 CPU processing is too slow, so all
> I have left is the nvcodec filters, plus I don't know any memory:GLMemory
> capable video sink other than glimagesink (glimagesinkelement didn't work,
> not sure why).
> 
> 2. sync=false
> 
> /gst-launch-1.0.exe filesrc location=4K.mkv ! matroskademux ! h265parse !
> nvh265dec ! glimagesink sync=false/
> 
> Plays back the video at high speed. Really smooth and fast, so I suspect
> it's not a performance issue I'm having. I've also tried to resizing the
> video stream (using gltransformation) to 2K, after the nvh265dec, that
> didn't change anything either.
> 
> Here are the things I've tried to improve the situation:
> 
> 3. vsync=true
> 
> /gst-launch-1.0.exe filesrc location=4K.mkv ! matroskademux ! h265parse !
> nvh265dec ! glimagesink sync=true vsync=true/

As Michael suggested, you can add a queue between vh265dec and glimagesink. Note
that the default maximum size is pretty small, so try adding it this way:

  ... ! nvh265dec ! queue max-size-bytes=0 ! glimagesink

Note that there is no "vsync" property on glimagesink in mainline GStreamer.
Please review all your applied patches as they might also be the cause of all
your troubles.

Another thing you may look at is the following fix that was merge recently:

https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/merge_requests/998

It's not much, but it removes 1 frame latency on the decoder. But this is
buffering latency, so on non-live playback, I have strong doubt it will make any
difference.

> 
> Not much changed, still small jumps from frame to frame.
> 
> 4. videorate
> 
> /gst-launch-1.0.exe filesrc location=4K.mkv ! matroskademux ! h265parse !
> nvh265dec ! queue ! videorate ! video/x-raw(ANY), framerate=60/1 !
> glimagesink sync=true vsync=true/
> 
> Or video/x-raw(memory:GLMemory) caps. This "feels" a bit better, smoother,
> but not perfect.
> 
> 5. videorate + sync off
> 
> /gst-launch-1.0.exe filesrc location=4K.mkv ! matroskademux ! h265parse !
> nvh265dec ! queue ! videorate ! video/x-raw(ANY), framerate=60/1 !
> glimagesink sync=false vsync=true/
> 
> I've seen another thread ( Choppy prores 4K playback
> <
> http://gstreamer-devel.966125.n4.nabble.com/Choppy-prores-4k-playback-td4691115.html
> > 
> ) in which it's mentioned that perhaps the individual buffers aren't
> processed fast enough at the sink - not sure if it applies to my case, but I
> assumed the above pipeline would avoid the problem of buffers being dropped
> because they were displayed too slowly... even if that's not correct, so far
> this has produced the best result, but it's still not as smooth as playing
> it back with MPC-BE.
> 
> I've been able to get so far with gst-launch, I'm thinking maybe I could get
> more information about what's really going on with programming this pipeline
> in C, but I'd be happy if someone wiser could give me some ideas as to which
> direction I should be going from here on.
> 
> 
> 
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel