Implementing Miracast?

Fri Dec 4 01:10:43 PST 2015

Sorry if this is completely off-topic, but could this be useful for
high performance screen recording also?
This seems to be all the rage on Windows these days, with software like
OBS (+AMD VCE support), Nvidia ShadowPlay (also HW accel encoding),
Xsplit, Twitch etc...

Regards
//Ernst

2015-12-04 9:07 GMT+01:00 Daniel Vetter <daniel at ffwll.ch>:
> On Thu, Dec 03, 2015 at 07:26:31PM +0200, Martin Peres wrote:
>> On 03/12/15 18:38, Ilia Mirkin wrote:
>> >On Thu, Dec 3, 2015 at 11:10 AM, Laurent Pinchart
>> ><laurent.pinchart at ideasonboard.com> wrote:
>> >>Hi Ilia,
>> >>
>> >>On Thursday 03 December 2015 11:03:28 Ilia Mirkin wrote:
>> >>>On Thu, Dec 3, 2015 at 10:53 AM, Laurent Pinchart wrote:
>> >>>>On Thursday 03 December 2015 10:42:50 Ilia Mirkin wrote:
>> >>>>>On Thu, Dec 3, 2015 at 10:34 AM, Laurent Pinchart wrote:
>> >>>>>>On Thursday 03 December 2015 14:42:51 Hannikainen, Jaakko wrote:
>> >>>>>>>Hello,
>> >>>>>>>
>> >>>>>>>We're developing Miracast (HDMI over Wireless connections). The
>> >>>>>>>current progress is that it 'works' in the userspace but doesn't have
>> >>>>>>>any integration with X/Wayland and can only mirror the current desktop
>> >>>>>>>using gstreamer.
>> >>>>>>>
>> >>>>>>>We're looking into extending the implementation so that we would be
>> >>>>>>>able to use the remote screens just as any other connected screen, but
>> >>>>>>>we're not quite sure where we should implement it.
>> >>>>>>>
>> >>>>>>>The DRM interface seems like the perfect fit since we wouldn't need to
>> >>>>>>>patch every compositor.
>> >>>>>>>
>> >>>>>>>Right now, gstreamer is the equivalent of the crtc/encoder, in the DRM
>> >>>>>>>model. Screens / crtcs are discovered using a WiFi's p2p protocol
>> >>>>>>>which means that screens should be hotpluggable. Since we cannot
>> >>>>>>>change the number of crtcs of a driver on the fly, we propose adding
>> >>>>>>>and removing gpus with one crtc attached and no rendering
>> >>>>>>>capabilities.
>> >>>>>>>
>> >>>>>>>Compositors and X currently use udev to list gpus and get run-time
>> >>>>>>>events for gpu hot-plugging (see the work from Dave Airlie for USB
>> >>>>>>>GPUs, using the modesetting X driver). We did not find a way to tell
>> >>>>>>>udev that we have a new device and it seems like the only way to get
>> >>>>>>>it to pick up our driver is from a uevent which can only be generated
>> >>>>>>>from the kernel.
>> >>>>>>>
>> >>>>>>>Since we have so many userspace components, it doesn't make sense to
>> >>>>>>>implement the entire driver in the kernel.
>> >>>>>>>
>> >>>>>>>We would thus need to have a communication from the kernel space to
>> >>>>>>>the userspace at least to send the flip commands to the fake crtc.
>> >>>>>>>Since we need this, why not implement everything in the userspace and
>> >>>>>>>just redirect the ioctls to the userspace driver?
>> >>>>>>>
>> >>>>>>>This is exactly what fuse / cuse [1] does, with the minor catch that
>> >>>>>>>it creates devices in /sys/class/cuse instead of drm. This prevents
>> >>>>>>>the wayland compositors and X to pick it up as a normal drm driver...
>> >>>>>>>
>> >>>>>>>We would thus need to have the drm subsystem create the device nodes
>> >>>>>>>for us when the userspace needs to create a new gpu. We could create a
>> >>>>>>>node named /dev/dri/cuse_card that, when opened, would allocate a node
>> >>>>>>>(/dev/dri/cardX) and would use cuse/fuse to redirect the ioctls to the
>> >>>>>>>process who opened /dev/dri/cuse_card.
>> >>>>>>>
>> >>>>>>>The process would then be responsible for decoding the ioctl and
>> >>>>>>>implementing the drm API.
>> >>>>>>>
>> >>>>>>>Since this is a major change which would allow proprietary drivers to
>> >>>>>>>be implemented in the userspace and since we may have missed something
>> >>>>>>>obvious, we would like to start a discussion on this. What are your
>> >>>>>>>thoughts?
>> >>>>>>As you raise the issue, how would you prevent proprietary userspace
>> >>>>>>drivers to be implemented ? Anything that would allow vendors to
>> >>>>>>destroy the Linux graphics ecosystem would receive a big nack from me.
>> >>>>>AFAIK the displaylink people already have precisely such a driver -- a
>> >>>>>(open-source) kernel module that allows their (closed-source)
>> >>>>>userspace blob to present a drm node to pass through modesetting/etc
>> >>>>>ioctl's.
>> >>>>Are you talking about the drivers/gpu/drm/udl/ driver ? I might be wrong
>> >>>>but I'm not aware of that kernel driver requiring a closed-source
>> >>>>userspace blob.
>> >>>Nope. That driver only works for their USB2 parts. This is what I mean:
>> >>>
>> >>>https://github.com/DisplayLink/evdi
>> >>>http://support.displaylink.com/knowledgebase/articles/679060
>> >>>http://support.displaylink.com/knowledgebase/articles/615714#ubuntu
>> >>Right. That's out-of-tree, people are free to screw up on their own there ;-)
>> >Sure, but it's identical to Jaakko's proposal from what I can
>> >(quickly) tell. And it's an example of someone taking an interface
>> >like that and writing a proprietary driver on top.
>> >
>> >   -ilia
>> >
>>
>> You are right Ilia, this is indeed what Jaakko and I had in mind, but they
>> did not re-use the fuse/cuse framework to do the serialization of the
>> ioctls.
>>
>> Not sure what we can do against allowing proprietary drivers to use this
>> feature though :s To be fair, nothing prevents any vendor to do this shim
>> themselves and nvidia definitely did it, and directly called their
>> closed-source driver.
>>
>> Any proposition on how to handle this case? I guess we could limit that to
>> screens only, no rendering. That would block any serious GPU manufacturer
>> from using this code even if any sane person would never write a driver in
>> the userspace...
>
> Hm for virtual devices like this I figured there's no point exporting the
> full kms api to userspace, but instead we'd just need a simple kms driver
> with just 1 crtc and 1 connector per drm_device. Plus a special device
> node (v4l is probably inappropriate since it doesn't do damage) where the
> miracast userspace can receive events with just the following information:
> - virtual screen size
> - fd to the underlying shmem node for the current fb. Or maybe a dma-buf
>   (but then we'd need the dma-buf mmap stuff to land first).
> - damage tracking
>
> If we want fancy, we could allow userspace to reply (through an ioctl)
> when it's done reading the previous image, which the kernel could then
> forward as vblank complete events.
>
> Connector configuration could be done by forcing the outputs (we'll send
> out uevents nowadays for that), so the only thing we need is some configfs
> to instantiate new copies of this.
>
> At least for miracst (as opposed to full-blown hw drivers in userspace) I
> don't think we need to export everything.
> Cheers, Daniel
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch
> _______________________________________________
> dri-devel mailing list
> dri-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/dri-devel