KMS to multiple displays, was Re: Building a graph for HDR10 output - stumped at sink
Nicolas Dufresne
nicolas at ndufresne.ca
Fri Apr 22 20:47:00 UTC 2022
Le jeudi 21 avril 2022 à 15:12 -0700, Bill Hofmann a écrit :
> Back again. Working on 1.20.1, I've modified kmssink to retrieve HDR caps from
> the stream and set an HDR Infoframe to the correct values, if present in the
> stream. This works like a charm if I display a stream on a single display, but
> if I try to display two streams on two displays (different connector-id and
> plane-id) from two separate processes OR from the same python program, I get a
> 13 Permission Denied from both the drmModeObjectSetProperty call and
> the drmModeSetPlane call, which seems nonsensical.
It is how DRM works, you can only have one DRM master. The DRM master is the
first authorized software that opens the device. In order to allow multiple
kmssink to work at the same time, they would have to share the DRM driver FD,
but also special care and perhaps locking may be needed. I think this could be
implemented with a GstContext and the associated sharing mechanism.
Though, to get out of the toy land, kmssink would need perhap to be ported to
use atomic API, which notably allow trying calls without affecting other
surfaces. That would build up to the need for a shared display manager (also
known as a compositor), specially to coordinate mode setting. Which mostly boils
down to, why not use a proper compositor ?
>
> When I comment out my DRM calls (to more or less get back to pre-
> mods), drmModeSetPlane still fails with the same error message (logs below,
> full attached)
>
> The graphs:
>
> GST_DEBUG=1,kms*:9 gst-launch-1.0 filesrc location=/home/wdh/test/0AB7525E-
> 6452-4A5B-8573-84125B5744B0/0/vw_0000.m4s ! parsebin ! vaapih265dec ! video/x-
> raw,format=P010_10LE ! queue max-size-bytes=100663300 ! kmssink restore-
> crtc=true connector-id=327 plane-id=100
>
> GST_DEBUG_FILE=kmslog.txt GST_DEBUG=1,kms*:9 gst-launch-1.0 filesrc
> location=/home/wdh/test/0AB7525E-6452-4A5B-8573-84125B5744B0/0/vw_0000.m4s !
> parsebin ! vaapih265dec ! video/x-raw,format=P010_10LE ! queue max-size-
> bytes=100663300 ! kmssink restore-crtc=true connector-id=308 plane-id=31
>
> I can run one or the other separately, so the connector-id and plane-id
> parameters are correct. But when I run the first, then immediately (from
> another shell, or by backgrounding the first) run the second, the log looks
> like this:
>
> 0:00:00.088549896 1815 0x564ce99f78c0 DEBUG kmssink
> gstkmssink.c:1403:gst_kms_sink_set_caps:<kmssink0> negotiated caps = video/x-
> raw, format=(string)P010_10LE, width=(int)3840, height=(int)2160, interlace-
> mode=(string)progressive, multiview-mode=(string)mono, multiview-
> flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-
> flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,
> pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)bt2100-pq,
> framerate=(fraction)60000/1001, mastering-display-
> info=(string)34000:16000:13250:34500:7500:3000:15635:16450:40000000:50,
> content-light-level=(string)9935:124
> 0:00:00.088563267 1815 0x564ce99f78c0 DEBUG kmssink
> gstkmssink.c:1449:gst_kms_sink_propose_allocation:<kmssink0> propose
> allocation
> 0:00:00.089074455 1815 0x564ce99f78c0 DEBUG kmsallocator
> gstkmsallocator.c:218:gst_kms_allocator_memory_create:<KMSMemory::allocator>
> Created BO plane 0 with stride 7680 and offset 0
> 0:00:00.089089090 1815 0x564ce99f78c0 DEBUG kmsallocator
> gstkmsallocator.c:218:gst_kms_allocator_memory_create:<KMSMemory::allocator>
> Created BO plane 1 with stride 7680 and offset 16588800
> 0:00:00.089098233 1815 0x564ce99f78c0 DEBUG kmsallocator
> gstkmsallocator.c:469:gst_kms_allocator_add_fb:<KMSMemory::allocator> bo
> handles: 1, 1, 0, 0
> 0:00:00.089116246 1815 0x564ce99f78c0 DEBUG kmsbufferpool
> gstkmsbufferpool.c:163:gst_kms_buffer_pool_alloc_buffer:<kmsbufferpool1>
> adding GstVideoMeta
> 0:00:00.089128011 1815 0x564ce99f78c0 DEBUG kmsallocator
> gstkmsallocator.c:114:gst_kms_allocator_memory_reset:<KMSMemory::allocator>
> removing fb id 368
> 0:00:00.089147324 1815 0x564ce99f78c0 DEBUG kmsallocator
> gstkmsallocator.c:218:gst_kms_allocator_memory_create:<KMSMemory::allocator>
> Created BO plane 0 with stride 7680 and offset 0
> 0:00:00.089165083 1815 0x564ce99f78c0 DEBUG kmsallocator
> gstkmsallocator.c:218:gst_kms_allocator_memory_create:<KMSMemory::allocator>
> Created BO plane 1 with stride 7680 and offset 16588800
> 0:00:00.089174233 1815 0x564ce99f78c0 DEBUG kmsallocator
> gstkmsallocator.c:469:gst_kms_allocator_add_fb:<KMSMemory::allocator> bo
> handles: 1, 1, 0, 0
> 0:00:00.089184435 1815 0x564ce99f78c0 DEBUG kmsbufferpool
> gstkmsbufferpool.c:163:gst_kms_buffer_pool_alloc_buffer:<kmsbufferpool1>
> adding GstVideoMeta
> 0:00:00.089192315 1815 0x564ce99f78c0 DEBUG kmsallocator
> gstkmsallocator.c:114:gst_kms_allocator_memory_reset:<KMSMemory::allocator>
> removing fb id 368
> 0:00:00.089216900 1815 0x564ce99f78c0 DEBUG kmsallocator
> gstkmsallocator.c:218:gst_kms_allocator_memory_create:<KMSMemory::allocator>
> Created BO plane 0 with stride 7680 and offset 0
> 0:00:00.089225922 1815 0x564ce99f78c0 DEBUG kmsallocator
> gstkmsallocator.c:218:gst_kms_allocator_memory_create:<KMSMemory::allocator>
> Created BO plane 1 with stride 7680 and offset 16588800
> 0:00:00.089234807 1815 0x564ce99f78c0 DEBUG kmsallocator
> gstkmsallocator.c:469:gst_kms_allocator_add_fb:<KMSMemory::allocator> bo
> handles: 1, 1, 0, 0
> 0:00:00.089243309 1815 0x564ce99f78c0 DEBUG kmsbufferpool
> gstkmsbufferpool.c:163:gst_kms_buffer_pool_alloc_buffer:<kmsbufferpool1>
> adding GstVideoMeta
> 0:00:00.138840940 1815 0x564ce99f78c0 TRACE kmssink
> gstkmssink.c:1848:gst_kms_sink_show_frame:<kmssink0> displaying fb 368
> 0:00:00.138860715 1815 0x564ce99f78c0 TRACE kmssink
> gstkmssink.c:1912:gst_kms_sink_show_frame:<kmssink0> drmModeSetPlane at (0,0)
> 3840x2160 sourcing at (0,0) 3840x2160
> 0:00:00.138871078 1815 0x564ce99f78c0 WARN kmssink
> gstkmssink.c:1922:gst_kms_sink_show_frame: drmModeSetPlane(368, 100, 167)
> failed -13, go to retry_set_plane
> 0:00:00.138880343 1815 0x564ce99f78c0 TRACE kmssink
> gstkmssink.c:1912:gst_kms_sink_show_frame:<kmssink0> drmModeSetPlane at (0,0)
> 3840x2160 sourcing at (0,0) 3840x2160
> 0:00:00.138888501 1815 0x564ce99f78c0 DEBUG kmssink
> gstkmssink.c:1971:gst_kms_sink_show_frame:<kmssink0> result = { 0, 0, 3840,
> 2160} / src = { 0, 0, 3840 2160 } / dst = { 0, 0, 3840 2160 }
> 0:00:00.138905801 1815 0x564ce99f78c0 WARN kmssink
> gstkmssink.c:1975:gst_kms_sink_show_frame:<kmssink0> error: drmModeSetPlane
> failed: Permission denied (13)
> ERROR: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: GStreamer
> encountered a general resource error.
> Additional debug info:
> ../subprojects/gst-plugins-bad/sys/kms/gstkmssink.c(1975):
> gst_kms_sink_show_frame (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:
> drmModeSetPlane failed: Permission denied (13)
> ERROR: pipeline doesn't want to preroll.
> 0:00:00.138981967 1815 0x564ce99f78c0 DEBUG kmsallocator
> gstkmsallocator.c:114:gst_kms_allocator_memory_reset:<KMSMemory::allocator>
> removing fb id 368
> Setting pipeline to NULL ...
> Freeing pipeline ...
>
> which propagates out and causes the playback to fail. Note I added the
> "retry_set_plane" logging as part of debugging.
>
> So - hoping this is something easy to solve, or at least *POSSIBLE* to solve.
> Thoughts?
>
> -Bill
>
>
>
>
> On Wed, Mar 30, 2022 at 2:12 PM Nicolas Dufresne <nicolas at ndufresne.ca> wrote:
> > Le mercredi 30 mars 2022 à 12:51 -0700, Bill Hofmann a écrit :
> > > Thanks for the hints on kmssink, sad it's not performant. Will consider.
> > >
> > > Looking at the waylandsink - it doesn't like P010 pixels, and doesn't seem
> > to
> > > want to negotiate NV12 pixels - so I'm concerned that it won't support
> > HDR10 -
> >
> > Yeah, HDR10 would require you to do some patching with early work. Also note
> > that I said in Weston, I don't expect this to work on Gnome or Plasma at the
> > moment. Regardless of the direction, you have to do some plumbing for HDR10
> > on
> > Linux.
> >
> > >
> > > GST_DEBUG=way*:5 gst-launch-1.0 filesrc
> > > location=04_Tryptic_Right_UHD_HDR10_010522.mp4 ! qtdemux ! h265parse !
> > > vaapih265dec ! video/x-raw,format=NV12 ! vaapipostproc ! waylandsink
> > > fullscreen=1
> > > Setting pipeline to PAUSED ...
> > > Pipeline is PREROLLING ...
> > > Got context from element 'vaapipostproc0': gst.gl.GLDisplay=context,
> > > gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\
> > gldisplaywayland0";
> > > Got context from element 'vaapipostproc0': gst.vaapi.Display=context,
> > > gst.vaapi.Display=(GstVaapiDisplay)"\(GstVaapiDisplayEGL\)\
> > vaapidisplayegl0";
> > > 0:00:00.103692914 6503 0x7f2270005b00 DEBUG waylandsink
> > > gstwaylandsink.c:512:gst_wayland_sink_get_caps:<waylandsink0> display
> > caps:
> > > video/x-raw, format=(string){ BGRA, BGRx, RGB16 }, width=(int)[ 1,
> > 2147483647
> > > ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1
> > ];
> > > video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16 },
> > width=(int)[
> > > 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1,
> > > 2147483647/1 ]
> > > 0:00:00.103860649 6503 0x7f2270005b00 DEBUG waylandsink
> > > gstwaylandsink.c:512:gst_wayland_sink_get_caps:<waylandsink0> display
> > caps:
> > > video/x-raw, format=(string){ BGRA, BGRx, RGB16 }, width=(int)[ 1,
> > 2147483647
> > > ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1
> > ];
> > > video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16 },
> > width=(int)[
> > > 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1,
> > > 2147483647/1 ]
> > > 0:00:00.104163879 6503 0x7f2270005b00 DEBUG waylandsink
> > > gstwaylandsink.c:512:gst_wayland_sink_get_caps:<waylandsink0> display
> > caps:
> > > video/x-raw, format=(string){ BGRA, BGRx, RGB16 }, width=(int)[ 1,
> > 2147483647
> > > ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1
> > ];
> > > video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16 },
> > width=(int)[
> > > 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1,
> > > 2147483647/1 ]
> > > 0:00:00.104282762 6503 0x7f2270005b00 DEBUG waylandsink
> > > gstwaylandsink.c:512:gst_wayland_sink_get_caps:<waylandsink0> display
> > caps:
> > > video/x-raw, format=(string){ BGRA, BGRx, RGB16 }, width=(int)[ 1,
> > 2147483647
> > > ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1
> > ];
> > > video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16 },
> > width=(int)[
> > > 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1,
> > > 2147483647/1 ]
> > > 0:00:00.104418197 6503 0x7f2270005b00 DEBUG waylandsink
> > > gstwaylandsink.c:512:gst_wayland_sink_get_caps:<waylandsink0> display
> > caps:
> > > video/x-raw, format=(string){ BGRA, BGRx, RGB16 }, width=(int)[ 1,
> > 2147483647
> > > ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1
> > ];
> > > video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16 },
> > width=(int)[
> > > 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1,
> > > 2147483647/1 ]
> > > 0:00:00.104696971 6503 0x7f2270005b00 DEBUG waylandsink
> > > gstwaylandsink.c:512:gst_wayland_sink_get_caps:<waylandsink0> display
> > caps:
> > > video/x-raw, format=(string){ BGRA, BGRx, RGB16 }, width=(int)[ 1,
> > 2147483647
> > > ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1
> > ];
> > > video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16 },
> > width=(int)[
> > > 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1,
> > > 2147483647/1 ]
> > > 0:00:00.104808871 6503 0x7f2270005b00 DEBUG waylandsink
> > > gstwaylandsink.c:512:gst_wayland_sink_get_caps:<waylandsink0> display
> > caps:
> > > video/x-raw, format=(string){ BGRA, BGRx, RGB16 }, width=(int)[ 1,
> > 2147483647
> > > ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1
> > ];
> > > video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16 },
> > width=(int)[
> > > 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1,
> > > 2147483647/1 ]
> > > 0:00:00.186018372 6503 0x7f2270005b00 DEBUG waylandsink
> > > gstwaylandsink.c:512:gst_wayland_sink_get_caps:<waylandsink0> display
> > caps:
> > > video/x-raw, format=(string){ BGRA, BGRx, RGB16 }, width=(int)[ 1,
> > 2147483647
> > > ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1
> > ];
> > > video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16 },
> > width=(int)[
> > > 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1,
> > > 2147483647/1 ]
> > > Redistribute latency...
> > > ERROR: from element /GstPipeline:pipeline0/GstQTDemux:qtdemux0: Internal
> > data
> > > stream error.
> > > Additional debug info:
> > > ../gst/isomp4/qtdemux.c(6545): gst_qtdemux_loop ():
> > > /GstPipeline:pipeline0/GstQTDemux:qtdemux0:
> > > streaming stopped, reason not-negotiated (-4)
> > > ERROR: pipeline doesn't want to preroll.
> > > Setting pipeline to NULL ...
> > > Freeing pipeline ...
> > > 0:00:00.190220809 6503 0x5562d2c0c230 DEBUG waylandsink
> > > gstwaylandsink.c:286:gst_wayland_sink_finalize:<waylandsink0> Finalizing
> > the
> > > sink..
> > >
> > > On Wed, Mar 30, 2022 at 4:50 AM Nicolas Dufresne <nicolas at ndufresne.ca>
> > wrote:
> > > >
> > > >
> > > > Le mar. 29 mars 2022 20 h 20, Bill Hofmann <bill.hofmann at gmail.com> a
> > > > écrit :
> > > > > Thanks for the very helpful feedback. I am able to connect to kmssink,
> > and
> > > > > after specifying connector-id and (the correct, I think) plane-id, I
> > get
> > > > > video out from the non-X view. Defaults don't work there, but that's
> > > > > manageable. Clearly not yet HDR (haven't made changes to kmssink
> > yet).
> > > > >
> > > > > HOWEVER, performance is exceptionally poor - whereas vaapisink is able
> > to
> > > > > keep up with 4k30 P010 without a problem (in a window on the desktop),
> > > > > kmssink seems to run at about 15fps. As someone relatively new to this
> > > > > whole process, what's the best way to debug these performance issues?
> > > > > Looking at debug all the way up to 9, there isn't (of course) any
> > helpful
> > > > > logged info at least that's helpful to me. I'll note autovideosink
> > also
> > > > > has performance issues (but again isn't full screen or non-X).
> > > >
> > > > There is poorly implemented vsync in kmssink, combined with DRM
> > emulation of
> > > > the legacy API kmssink uses, the frame rate get halved. The only
> > solution I
> > > > have for now is to comment out the legacy vsync code, the emulation
> > will
> > > > sync already.
> > > >
> > > > The use a queue before the sink, ensure the byte size is large enough
> > for
> > > > 4K.
> > > >
> > > > Long term solution is to move on to atomic API. There is a MR for that,
> > but
> > > > last time I checked it was of poor quality and the author wasn't
> > replying.
> > > >
> > > > >
> > > > > A little more context - this is an "embedded" Linux situation - will
> > never
> > > > > be other than full screen video (likely no UI), so there is no need
> > for
> > > > > interop with desktop functionality.
> > > >
> > > > I would really like to see this type of use cases improve. This is
> > common in
> > > > NXP and Xilinx SDK, though they hold downstream patches to workaround
> > this.
> > > >
> > > > Meanwhile, be aware that Weston + waylandsink works better, Weston
> > > > automatically (and smartly) use HW layer. The limitation is that it only
> > > > works in combination of a GPU, it's not supported by the pixman backend
> > > > (yet?).
> > > >
> > > > >
> > > > > -Bill
> > > > >
> > > > > On Sun, Mar 27, 2022 at 6:05 AM Nicolas Dufresne
> > <nicolas at ndufresne.ca>
> > > > > wrote:
> > > > > >
> > > > > >
> > > > > > Le sam. 26 mars 2022 21 h 45, Bill Hofmann via gstreamer-devel
> > > > > > <gstreamer-devel at lists.freedesktop.org> a écrit :
> > > > > > >
> > > > > > > So. What's the next step here? Is this a big gap in kmssink? Is
> > there
> > > > > > > another sink I should be trying?
> > > > > >
> > > > > > There is no mainline HDR10 support on Linux outside of vendors BSP
> > (NXP
> > > > > > to note one vendor). Vendors seems to want to compete on this
> > feature,
> > > > > > and don't really work very hard to come up with generic
> > implementation.
> > > > > >
> > > > > > That being said, if the kmssink is sufficient for your use case,
> > then
> > > > > > its the smallest way forward, since you don't need a new Wayland
> > > > > > protocol and compositors support. What I'm uncertain though is if
> > this
> > > > > > will work with the fact the kmssink code base have aged and isn't
> > using
> > > > > > the newest DRM API (atomic kms). But other then that, if you have
> > done
> > > > > > that in the past, this is plain C and mapping the caps field to the
> > DRM
> > > > > > properties is all you need. Unlike HDR10+, the metadata does not
> > change
> > > > > > every frames.
> > > > > >
> > > > > >
> > > > > > Regards,
> > > > > > Nicolas
> > > > > >
> > > > > > p.s. vaapipostproc have a HDR to SDR converter to help improve SDR
> > > > > > output quality
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Bill Hofmann
> > > > > +1 510 387-0952
> > >
> > >
> >
>
>
More information about the gstreamer-devel
mailing list