Building a graph for HDR10 output - stumped at sink

Wed Mar 30 21:12:38 UTC 2022

Le mercredi 30 mars 2022 à 12:51 -0700, Bill Hofmann a écrit :
> Thanks for the hints on kmssink, sad it's not performant. Will consider.
> 
> Looking at the waylandsink - it doesn't like P010 pixels, and doesn't seem to
> want to negotiate NV12 pixels - so I'm concerned that it won't support HDR10 -

Yeah, HDR10 would require you to do some patching with early work. Also note
that I said in Weston, I don't expect this to work on Gnome or Plasma at the
moment.  Regardless of the direction, you have to do some plumbing for HDR10 on
Linux.

> 
> GST_DEBUG=way*:5 gst-launch-1.0 filesrc
> location=04_Tryptic_Right_UHD_HDR10_010522.mp4 ! qtdemux ! h265parse !
> vaapih265dec ! video/x-raw,format=NV12 ! vaapipostproc ! waylandsink
> fullscreen=1
> Setting pipeline to PAUSED ...
> Pipeline is PREROLLING ...
> Got context from element 'vaapipostproc0': gst.gl.GLDisplay=context,
> gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\ gldisplaywayland0";
> Got context from element 'vaapipostproc0': gst.vaapi.Display=context,
> gst.vaapi.Display=(GstVaapiDisplay)"\(GstVaapiDisplayEGL\)\ vaapidisplayegl0";
> 0:00:00.103692914  6503 0x7f2270005b00 DEBUG            waylandsink
> gstwaylandsink.c:512:gst_wayland_sink_get_caps:<waylandsink0> display caps:
> video/x-raw, format=(string){ BGRA, BGRx, RGB16 }, width=(int)[ 1, 2147483647
> ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ];
> video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16 }, width=(int)[
> 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1,
> 2147483647/1 ]
> 0:00:00.103860649  6503 0x7f2270005b00 DEBUG            waylandsink
> gstwaylandsink.c:512:gst_wayland_sink_get_caps:<waylandsink0> display caps:
> video/x-raw, format=(string){ BGRA, BGRx, RGB16 }, width=(int)[ 1, 2147483647
> ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ];
> video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16 }, width=(int)[
> 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1,
> 2147483647/1 ]
> 0:00:00.104163879  6503 0x7f2270005b00 DEBUG            waylandsink
> gstwaylandsink.c:512:gst_wayland_sink_get_caps:<waylandsink0> display caps:
> video/x-raw, format=(string){ BGRA, BGRx, RGB16 }, width=(int)[ 1, 2147483647
> ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ];
> video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16 }, width=(int)[
> 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1,
> 2147483647/1 ]
> 0:00:00.104282762  6503 0x7f2270005b00 DEBUG            waylandsink
> gstwaylandsink.c:512:gst_wayland_sink_get_caps:<waylandsink0> display caps:
> video/x-raw, format=(string){ BGRA, BGRx, RGB16 }, width=(int)[ 1, 2147483647
> ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ];
> video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16 }, width=(int)[
> 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1,
> 2147483647/1 ]
> 0:00:00.104418197  6503 0x7f2270005b00 DEBUG            waylandsink
> gstwaylandsink.c:512:gst_wayland_sink_get_caps:<waylandsink0> display caps:
> video/x-raw, format=(string){ BGRA, BGRx, RGB16 }, width=(int)[ 1, 2147483647
> ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ];
> video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16 }, width=(int)[
> 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1,
> 2147483647/1 ]
> 0:00:00.104696971  6503 0x7f2270005b00 DEBUG            waylandsink
> gstwaylandsink.c:512:gst_wayland_sink_get_caps:<waylandsink0> display caps:
> video/x-raw, format=(string){ BGRA, BGRx, RGB16 }, width=(int)[ 1, 2147483647
> ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ];
> video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16 }, width=(int)[
> 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1,
> 2147483647/1 ]
> 0:00:00.104808871  6503 0x7f2270005b00 DEBUG            waylandsink
> gstwaylandsink.c:512:gst_wayland_sink_get_caps:<waylandsink0> display caps:
> video/x-raw, format=(string){ BGRA, BGRx, RGB16 }, width=(int)[ 1, 2147483647
> ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ];
> video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16 }, width=(int)[
> 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1,
> 2147483647/1 ]
> 0:00:00.186018372  6503 0x7f2270005b00 DEBUG            waylandsink
> gstwaylandsink.c:512:gst_wayland_sink_get_caps:<waylandsink0> display caps:
> video/x-raw, format=(string){ BGRA, BGRx, RGB16 }, width=(int)[ 1, 2147483647
> ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ];
> video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16 }, width=(int)[
> 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1,
> 2147483647/1 ]
> Redistribute latency...
> ERROR: from element /GstPipeline:pipeline0/GstQTDemux:qtdemux0: Internal data
> stream error.
> Additional debug info:
> ../gst/isomp4/qtdemux.c(6545): gst_qtdemux_loop ():
> /GstPipeline:pipeline0/GstQTDemux:qtdemux0:
> streaming stopped, reason not-negotiated (-4)
> ERROR: pipeline doesn't want to preroll.
> Setting pipeline to NULL ...
> Freeing pipeline ...
> 0:00:00.190220809  6503 0x5562d2c0c230 DEBUG            waylandsink
> gstwaylandsink.c:286:gst_wayland_sink_finalize:<waylandsink0> Finalizing the
> sink..
> 
> On Wed, Mar 30, 2022 at 4:50 AM Nicolas Dufresne <nicolas at ndufresne.ca> wrote:
> > 
> > 
> > Le mar. 29 mars 2022 20 h 20, Bill Hofmann <bill.hofmann at gmail.com> a
> > écrit :
> > > Thanks for the very helpful feedback. I am able to connect to kmssink, and
> > > after specifying connector-id and (the correct, I think) plane-id, I get
> > > video out from the non-X view. Defaults don't work there, but that's
> > > manageable. Clearly not yet HDR (haven't made changes to kmssink yet). 
> > > 
> > > HOWEVER, performance is exceptionally poor - whereas vaapisink is able to
> > > keep up with 4k30 P010 without a problem (in a window on the desktop),
> > > kmssink seems to run at about 15fps. As someone relatively new to this
> > > whole process, what's the best way to debug these performance issues? 
> > > Looking at debug all the way up to 9, there isn't (of course) any helpful
> > > logged info at least that's helpful to me.  I'll note autovideosink also
> > > has performance issues (but again isn't full screen or non-X).
> > 
> > There is poorly implemented vsync in kmssink, combined with DRM emulation of
> > the legacy API kmssink uses, the frame rate get halved. The only solution I
> > have for now is to comment out the legacy vsync  code, the emulation will
> > sync already.
> > 
> > The use a queue before the sink, ensure the byte size is large enough for
> > 4K.
> > 
> > Long term solution is to move on to atomic API. There is a MR for that, but
> > last time I checked it was of poor quality and the author wasn't replying.
> > 
> > > 
> > > A little more context - this is an "embedded" Linux situation - will never
> > > be other than full screen video (likely no UI), so there is no need for
> > > interop with desktop functionality.
> > 
> > I would really like to see this type of use cases improve. This is common in
> > NXP and Xilinx SDK, though they hold downstream patches to workaround this.
> > 
> > Meanwhile, be aware that Weston + waylandsink works better, Weston
> > automatically (and smartly) use HW layer. The limitation is that it only
> > works in combination of a GPU, it's not supported by the pixman backend
> > (yet?).
> > 
> > > 
> > > -Bill
> > > 
> > > On Sun, Mar 27, 2022 at 6:05 AM Nicolas Dufresne <nicolas at ndufresne.ca>
> > > wrote:
> > > > 
> > > > 
> > > > Le sam. 26 mars 2022 21 h 45, Bill Hofmann via gstreamer-devel
> > > > <gstreamer-devel at lists.freedesktop.org> a écrit :
> > > > > 
> > > > > So.  What's the next step here? Is this a big gap in kmssink? Is there
> > > > > another sink I should be trying?
> > > > 
> > > > There is no mainline HDR10 support on Linux outside of vendors BSP (NXP
> > > > to note one vendor). Vendors seems to want to compete on this feature,
> > > > and don't really work very hard to come up with generic implementation.
> > > > 
> > > > That being said, if the kmssink is sufficient for your use case, then
> > > > its the smallest way forward, since you don't need a new Wayland
> > > > protocol and compositors support. What I'm uncertain though is if this
> > > > will work with the fact the kmssink code base have aged and isn't using
> > > > the newest DRM API (atomic kms). But other then that, if you have done
> > > > that in the past, this is plain C and mapping the caps field to the DRM
> > > > properties is all you need. Unlike HDR10+, the metadata does not change
> > > > every frames.
> > > > 
> > > > 
> > > > Regards,
> > > > Nicolas
> > > > 
> > > > p.s. vaapipostproc have a HDR to SDR converter to help improve SDR
> > > > output quality
> > > > 
> > > 
> > > 
> > > -- 
> > > Bill Hofmann
> > > +1 510 387-0952
> 
> 