[PATCH 00/10] Generic USB Display driver

[Noralf Trønnes]

Den 29.04.2020 14.48, skrev Noralf Trønnes:
> Hi,
> 
> A while back I had the idea to turn a Raspberry Pi Zero into a $5
> USB to HDMI/SDTV/DSI/DPI display adapter.
> 
> This series adds a USB host driver and a device/gadget driver to achieve
> that.
> 
> The reason for calling it 'Generic' is so anyone can make a USB
> display/adapter against this driver, all that's needed is to add a USB
> vid:pid. I was hoping to have someone working on a microcontroller based
> USB display by now, but unfortunately that has been delayed. It would
> have been nice to have a microcontroller implementation to ensure that I
> haven't made things unnecessary difficult to implement.
> 
> Performance
> The one thing that decides how useful this all is, is how smooth an
> experience it gives. My hope was that it should not be noticeably laggy
> with ordinary office use on 1920x1080 at RG16. I'm pleased to see that it's
> also possible to watch youtube movies, although not in fullscreen.
> 
> Some of the main factors that affects performance:
> - Display resolution
> - Userspace providing damage reports (FB_DAMAGE_CLIPS or
> DRM_IOCTL_MODE_DIRTYFB)
> - Color depth (DRM_CAP_DUMB_PREFERRED_DEPTH = 16 if RGB565)
> - How well the frames compress (lz4)
> - Gadget device memory bandwidth, CPU power for decompression
> - (Big endian hosts will have to do byte swapping on the frames)

One factor that I forgot is USB2 vs USB3.
The Pi's have a USB2 Device controller (dwc2). I couldn't find a cheap
board with a USB3 Device controller that could run mainline Linux, so I
haven't tried that.

> 
> I've tested these:
> - xorg-server on Pi4. This was nice and smooth since it uses
> DRM_IOCTL_MODE_DIRTYFB and honours DRM_CAP_DUMB_PREFERRED_DEPTH.
> - Ubuntu 20.04 GNOME on x86. This was useable, but not so good for
> movies. GNOME doesn't look at DRM_CAP_DUMB_PREFERRED_DEPTH and doesn't
> set FB_DAMAGE_CLIPS on the pageflips.
> 
> I've made a short video to show what it looks like[1].

I got a question if this would work with usbip[4], USB over IP.
I did a quick test with two Pi4's connected by cable to the same network
switch (1Gb). Showing a movie in a window like my previous test didn't
show much of a difference. Maybe some occasional glitching, hard to tell
without proper tests.

There's no pageflipping on the device side, so it could be tearing that
I saw.

Noralf.

[4] tools/usb/usbip/README

> 
> I have used a Pi4 as the gadget during development since it has much
> better memory bandwith (4000 vs 200 MBps)[2] and CPU than the PiZ. They
> both have the same gadget controller (dwc2).
> 
> I did an RFC [3] of this 2 months ago where I looked at doing a Multi
> Function USB device. I gave up on that when I realised how much work the
> review process would be. So I stripped down to my original idea. I have
> made sure that the drivers will tolerate another USB interface of type
> VENDOR, so it's still possible for the display to be part of a multi
> function device.
> 
> Noralf.
> 
> [1] https://youtu.be/AhGZWwUm8JU
> [2] https://magpi.raspberrypi.org/articles/raspberry-pi-specs-benchmarks
> [3] https://patchwork.freedesktop.org/series/73508/
>