How to handle pixel format synonyms ("YUYV" is also known as "YUY2")?

Wed Aug 29 11:47:58 UTC 2018

Le mercredi 29 août 2018 à 02:28 +0000, Matthew Thyer a écrit :
> I'm working on an application to serve hardware accelerated H.264/AVC
> video via RTSP. I'm using the GStreamer RTSP Server 1.14.2 (built
> from source) and (currently) a Video 4 Linux 2 source.
>  
> My current problem is in trying to validate whether a user specified
> pixel format is supported by the camera. I see that can could either
> enumerate the supported formats or just ask the device to try
> (VIDIOC_TRY_FMT) or set (VIDIOC_S_FMT) a particular format. However,
> when I try (VIDIOC_TRY_FMT) any, garbage, made up, pixel format (e.g.
> "ifsg"), the ioctl succeeds! The driver seems to pick some other
> format (maybe it's default).
>  
> The only way I know that the chosen format doesn't match what the
> user requested is to read the pixel format out of the format
> structure (fmt.pix.pixelformat or fmt.pix_mp.pixelformat) and compare
> it with what I had requested.
>  
> This would be fine excepting for the problem of pixel format synonyms
> (i.e. in particular, "YUYV" is also known as "YUY2").

There is no standard for fourcc, as a side effect, the GStreamer chosen
one does not match the V4L2 one. Though, this does not matter, the
translation is handled by GStreamer. Instead of calling into V4L2 API
to enumerate the formats, you should use the GstDeviceMonitor API to
track and enumerate cameras. This way, you're application becomes V4L2
agnostic, and a transition to pipewire could be made possible.

>  
> I'm testing with Ubuntu 18.04.1 LTS x86_64 (bionic) with its standard
> kernel (4.15.0-33-generic) and the "Logitech HD Webcam C270".
> (Note that I've built GStreamer RTSP server v1.14.2 from its source
> release and can provide my configure line if required).
>  
> The V4L2 driver for this camera reports the following info and pixel
> formats:
>  
> $ v4l2-ctl --info --list-formats
> Driver Info (not using libv4l2):
>         Driver name   : uvcvideo
>         Card type     : UVC Camera (046d:0825)
>         Bus info      : usb-0000:00:14.0-1
>         Driver version: 4.15.18
>         Capabilities  : 0x84200001
>                 Video Capture
>                 Streaming
>                 Extended Pix Format
>                 Device Capabilities
>         Device Caps   : 0x04200001
>                 Video Capture
>                 Streaming
>                 Extended Pix Format
> ioctl: VIDIOC_ENUM_FMT
>         Index       : 0
>         Type        : Video Capture
>         Pixel Format: 'YUYV'
>         Name        : YUYV 4:2:2
>  
>         Index       : 1
>         Type        : Video Capture
>         Pixel Format: 'MJPG' (compressed)
>         Name        : Motion-JPEG
>  
> Note that the camera doesn’t advertise support for the “YUY2” pixel
> format even though it’s identical to the “YUYV” pixel format!
>  
> For this camera, my GStreamer RTSP server pipeline should be
> something like:
> v4l2src ! video/x-raw,format=YUYV,width=800,height=600,framerate=20/1 
> ! videoconvert ! x264enc ! video/x-h264,profile=high ! rtph264pay
> name=pay0 pt=96
>  
> This should work as 800x600 at 20/1 with pixelformat=YUYV is supported
> by the camera:
> $ v4l2-ctl --list-
> frameintervals=width=800,height=600,pixelformat=YUYV
> ioctl: VIDIOC_ENUM_FRAMEINTERVALS
>         Interval: Discrete 0.050s (20.000 fps)
>         Interval: Discrete 0.067s (15.000 fps)
>         Interval: Discrete 0.100s (10.000 fps)
>         Interval: Discrete 0.200s (5.000 fps)
>  
> Unfortunately the x264enc encoder doesn't support the YUYV
> pixelformat. It's sink pad only supports the following formats:
> (string)Y444, (string)Y42B, (string)I420, (string)YV12, (string)NV12
>  
> This is why I've included the videoconvert (Colorspace converter)
> element in my pipeline. However, the pipeline still fails because the
> videoconvert element sink pad also doesn't support the YUYV
> pixelformat!
> It only supports:
> (string)I420, (string)YV12, (string)YUY2, (string)UYVY, (string)AYUV,
> (string)RGBx, (string)BGRx, (string)xRGB, (string)xBGR, (string)RGBA,
> (string)BGRA, (string)ARGB, (string)ABGR, (string)RGB, (string)BGR,
> (string)Y41B, (string)Y42B, (string)YVYU, (string)Y444, (string)v210,
> (string)v216, (string)NV12, (string)NV21, (string)GRAY8,
> (string)GRAY16_BE, (string)GRAY16_LE, (string)v308, (string)RGB16,
> (string)BGR16, (string)RGB15, (string)BGR15, (string)UYVP,
> (string)A420, (string)RGB8P, (string)YUV9, (string)YVU9,
> (string)IYU1, (string)ARGB64, (string)AYUV64, (string)r210,
> (string)I420_10BE, (string)I420_10LE, (string)I422_10BE,
> (string)I422_10LE, (string)Y444_10BE, (string)Y444_10LE, (string)GBR,
> (string)GBR_10BE, (string)GBR_10LE, (string)NV16, (string)NV24,
> (string)NV12_64Z32, (string)A420_10BE, (string)A420_10LE,
> (string)A422_10BE, (string)A422_10LE, (string)A444_10BE,
> (string)A444_10LE, (string)NV61, (string)P010_10BE,
> (string)P010_10LE, (string)IYU2, (string)VYUY, (string)GBRA,
> (string)GBRA_10BE, (string)GBRA_10LE, (string)GBR_12BE,
> (string)GBR_12LE, (string)GBRA_12BE, (string)GBRA_12LE,
> (string)I420_12BE, (string)I420_12LE, (string)I422_12BE,
> (string)I422_12LE, (string)Y444_12BE, (string)Y444_12LE,
> (string)GRAY10_LE32, (string)NV12_10LE32, (string)NV16_10LE32
>  
> Searching around a bit, I found that pixelformat YUYV is also known
> as YUY2 and when I try that it all works!
> i.e. this pipeline works when I try it with the GStreamer RTSP server
> test-readme example:
> v4l2src ! video/x-raw,format=YUY2,width=800,height=600,framerate=20/1 
> ! videoconvert ! x264enc ! video/x-h264,profile=high ! rtph264pay
> name=pay0 pt=96
>  
> Why does this work when the driver doesn't advertise support for a
> format named "YUY2"?
>  
> What I think may be happening is that the driver is using it's
> default "YUYV" format when asked for a format that it doesn't support
> and GStreamer RTSP is not checking whether the set format matched
> what was requested. Videoconvert is happy because it thinks it's
> getting YUY2 and it is because the driver's default of YUYV is the
> same as YUY2.
>  
> I need to work out how to programmatically validate the user's
> requested pixel format and respond with meaningful error messages
> when things don't work.
>  
> Ideally, the V4L2 driver should list support for all the synonyms of
> a pixel format that it supports.
>  
> Please help!
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel