How to handle pixel format synonyms ("YUYV" is also known as "YUY2")?

Matthew Thyer matthew.thyer at adelaide.edu.au
Wed Aug 29 02:28:24 UTC 2018


I'm working on an application to serve hardware accelerated H.264/AVC video via RTSP. I'm using the GStreamer RTSP Server 1.14.2 (built from source) and (currently) a Video 4 Linux 2 source.

My current problem is in trying to validate whether a user specified pixel format is supported by the camera. I see that can could either enumerate the supported formats or just ask the device to try (VIDIOC_TRY_FMT) or set (VIDIOC_S_FMT) a particular format. However, when I try (VIDIOC_TRY_FMT) any, garbage, made up, pixel format (e.g. "ifsg"), the ioctl succeeds! The driver seems to pick some other format (maybe it's default).

The only way I know that the chosen format doesn't match what the user requested is to read the pixel format out of the format structure (fmt.pix.pixelformat or fmt.pix_mp.pixelformat) and compare it with what I had requested.

This would be fine excepting for the problem of pixel format synonyms (i.e. in particular, "YUYV" is also known as "YUY2").

I'm testing with Ubuntu 18.04.1 LTS x86_64 (bionic) with its standard kernel (4.15.0-33-generic) and the "Logitech HD Webcam C270".
(Note that I've built GStreamer RTSP server v1.14.2 from its source release and can provide my configure line if required).

The V4L2 driver for this camera reports the following info and pixel formats:

$ v4l2-ctl --info --list-formats
Driver Info (not using libv4l2):
        Driver name   : uvcvideo
        Card type     : UVC Camera (046d:0825)
        Bus info      : usb-0000:00:14.0-1
        Driver version: 4.15.18
        Capabilities  : 0x84200001
                Video Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04200001
                Video Capture
                Streaming
                Extended Pix Format
ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'YUYV'
        Name        : YUYV 4:2:2

        Index       : 1
        Type        : Video Capture
        Pixel Format: 'MJPG' (compressed)
        Name        : Motion-JPEG

Note that the camera doesn't advertise support for the "YUY2" pixel format even though it's identical to the "YUYV" pixel format!

For this camera, my GStreamer RTSP server pipeline should be something like:
v4l2src ! video/x-raw,format=YUYV,width=800,height=600,framerate=20/1 ! videoconvert ! x264enc ! video/x-h264,profile=high ! rtph264pay name=pay0 pt=96

This should work as 800x600 at 20/1 with pixelformat=YUYV is supported by the camera:
$ v4l2-ctl --list-frameintervals=width=800,height=600,pixelformat=YUYV
ioctl: VIDIOC_ENUM_FRAMEINTERVALS
        Interval: Discrete 0.050s (20.000 fps)
        Interval: Discrete 0.067s (15.000 fps)
        Interval: Discrete 0.100s (10.000 fps)
        Interval: Discrete 0.200s (5.000 fps)

Unfortunately the x264enc encoder doesn't support the YUYV pixelformat. It's sink pad only supports the following formats:
(string)Y444, (string)Y42B, (string)I420, (string)YV12, (string)NV12

This is why I've included the videoconvert (Colorspace converter) element in my pipeline. However, the pipeline still fails because the videoconvert element sink pad also doesn't support the YUYV pixelformat!
It only supports:
(string)I420, (string)YV12, (string)YUY2, (string)UYVY, (string)AYUV, (string)RGBx, (string)BGRx, (string)xRGB, (string)xBGR, (string)RGBA, (string)BGRA, (string)ARGB, (string)ABGR, (string)RGB, (string)BGR, (string)Y41B, (string)Y42B, (string)YVYU, (string)Y444, (string)v210, (string)v216, (string)NV12, (string)NV21, (string)GRAY8, (string)GRAY16_BE, (string)GRAY16_LE, (string)v308, (string)RGB16, (string)BGR16, (string)RGB15, (string)BGR15, (string)UYVP, (string)A420, (string)RGB8P, (string)YUV9, (string)YVU9, (string)IYU1, (string)ARGB64, (string)AYUV64, (string)r210, (string)I420_10BE, (string)I420_10LE, (string)I422_10BE, (string)I422_10LE, (string)Y444_10BE, (string)Y444_10LE, (string)GBR, (string)GBR_10BE, (string)GBR_10LE, (string)NV16, (string)NV24, (string)NV12_64Z32, (string)A420_10BE, (string)A420_10LE, (string)A422_10BE, (string)A422_10LE, (string)A444_10BE, (string)A444_10LE, (string)NV61, (string)P010_10BE, (string)P010_10LE, (string)IYU2, (string)VYUY, (string)GBRA, (string)GBRA_10BE, (string)GBRA_10LE, (string)GBR_12BE, (string)GBR_12LE, (string)GBRA_12BE, (string)GBRA_12LE, (string)I420_12BE, (string)I420_12LE, (string)I422_12BE, (string)I422_12LE, (string)Y444_12BE, (string)Y444_12LE, (string)GRAY10_LE32, (string)NV12_10LE32, (string)NV16_10LE32

Searching around a bit, I found that pixelformat YUYV is also known as YUY2 and when I try that it all works!
i.e. this pipeline works when I try it with the GStreamer RTSP server test-readme example:
v4l2src ! video/x-raw,format=YUY2,width=800,height=600,framerate=20/1 ! videoconvert ! x264enc ! video/x-h264,profile=high ! rtph264pay name=pay0 pt=96

Why does this work when the driver doesn't advertise support for a format named "YUY2"?

What I think may be happening is that the driver is using it's default "YUYV" format when asked for a format that it doesn't support and GStreamer RTSP is not checking whether the set format matched what was requested. Videoconvert is happy because it thinks it's getting YUY2 and it is because the driver's default of YUYV is the same as YUY2.

I need to work out how to programmatically validate the user's requested pixel format and respond with meaningful error messages when things don't work.

Ideally, the V4L2 driver should list support for all the synonyms of a pixel format that it supports.

Please help!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20180829/ca86ab92/attachment-0001.html>


More information about the gstreamer-devel mailing list