[RFC] Standardize YUV support in the fbdev API
Andy Walls
awalls at md.metrocast.net
Tue May 17 17:21:07 PDT 2011
On Wed, 2011-05-18 at 00:07 +0200, Laurent Pinchart wrote:
> Hi everybody,
>
> I need to implement support for a YUV frame buffer in an fbdev driver. As the
> fbdev API doesn't support this out of the box, I've spent a couple of days
> reading fbdev (and KMS) code and thinking about how we could cleanly add YUV
> support to the API. I'd like to share my findings and thoughts, and hopefully
> receive some comments back.
I haven't looked at anything below, but I'll mention that the ivtv
driver presents an fbdev interface for the YUV output stream of the
CX23415.
It may be worth a look and asking Hans what are the short-comings.
-Andy
> The terms 'format', 'pixel format', 'frame buffer format' and 'data format'
> will be used interchangeably in this e-mail. They all refer to the way pixels
> are stored in memory, including both the representation of a pixel as integer
> values and the layout of those integer values in memory.
>
>
> History and current situation
> -----------------------------
>
> The fbdev API predates YUV frame buffers. In those old days developers only
> cared (and probably thought) about RGB frame buffers, and they developed an
> API that could express all kind of weird RGB layout in memory (such as R-
> GGGGGGGGGGG-BBBB for instance, I can't imagine hardware implementing that).
> This resulted in individual bit field description for the red, green, blue and
> alpha components:
>
> struct fb_bitfield {
> __u32 offset; /* beginning of bitfield */
> __u32 length; /* length of bitfield */
> __u32 msb_right; /* != 0 : Most significant bit is */
> /* right */
> };
>
> Grayscale formats were pretty common, so a grayscale field tells color formats
> (grayscale == 0) from grayscale formats (grayscale != 0).
>
> People already realized that hardware developers were crazily inventive (the
> word to remember here is crazily), and that non-standard formats would be
> needed at some point. The fb_var_screeninfo ended up containing the following
> format-related fields.
>
> struct fb_var_screeninfo {
> ...
> __u32 bits_per_pixel; /* guess what */
> __u32 grayscale; /* != 0 Graylevels instead of colors */
>
> struct fb_bitfield red; /* bitfield in fb mem if true color, */
> struct fb_bitfield green; /* else only length is significant */
> struct fb_bitfield blue;
> struct fb_bitfield transp; /* transparency */
>
> __u32 nonstd; /* != 0 Non standard pixel format */
> ...
> };
>
> Two bits have been specified for the nonstd field:
>
> #define FB_NONSTD_HAM 1 /* Hold-And-Modify (HAM) */
> #define FB_NONSTD_REV_PIX_IN_B 2 /* order of pixels in each byte is reversed
> */
>
> The FB_NONSTD_HAM bit is used by the video/amifb.c driver only to enable HAM
> mode[1]. I very much doubt that any new hardware will implement HAM mode (and
> I certainly hope none will).
>
> The FB_NONSTD_REV_PIX_IN_B is used in video/fb_draw.h by the generic bitblit,
> fillrect and copy area implementations, not directly by drivers.
>
> A couple of drivers hardcode the nonstd field to 1 for some reason. Those are
> video/arcfb.c (1bpp gray display), video/hecubafb.c (1bpp gray display) and
> video/metronomefb.c (8bpp gray display).
>
> The following drivers use nonstd for various other (and sometimes weird)
> purposes:
>
> video/arkfb.c
> Used in 4bpp mode only, to control fb_setcolreg operation
> video/fsl-diu-fb.c
> Set var->nonstd bits into var->sync (why?)
> video/pxafb.c
> Encode frame buffer xpos and ypos in the nonstd field
> video/s3fb.c
> Used in 4bpp mode only, to control fb_setcolreg operation
> video/vga16fb.c
> When panning in non-8bpp, non-text mode, decrement xoffset
> Do some other weird stuff when not 0
> video/i810/i810_main.c
> Select direct color mode when set to 1 (truecolor otherwise)
>
> Fast forward a couple of years, hardware provides support for YUV frame
> buffers. Several drivers had to provide YUV format selection to applications,
> without any standard API to do so. All of them used the nonstd field for that
> purpose:
>
> media/video/ivtv/
> Enable YUV mode when set to 1
> video/pxafb.c
> Encode pixel format in the nonstd field
> video/sh_mobile_lcdfb.c
> If bpp == 12 and nonstd != 0, enable NV12 mode
> If bpp == 16 or bpp == 24, ?
> video/omap/omapfb_main.c
> Select direct color mode when set to 1 (depend on bpp otherwise)
> Used as a pixel format identifier (YUV422, YUV420 or YUY422)
> video/omap2/omapfb/omapfb-main.c
> Select direct color mode when set to 1 (depend on bpp otherwise)
> Used as a pixel format identifier (YUV422 or YUY422)
>
> Those drivers use the nonstd field in different, incompatible ways.
>
>
> Other related APIs
> ------------------
>
> Beside the fbdev API, two other kernel/userspace APIs deal with video-related
> modes and formats.
>
> - Kernel Mode Setting (KMS)
>
> The KMS API is similar in purpose to XRandR. Its main purpose is to provide a
> kernel API to configure output video modes. As such, it doesn't care about
> frame buffer formats, as they are irrelevant at the CRTC output.
>
> In addition to handling video modes, the KMS API also provides support for
> creating (and managing) frame buffer devices, as well as dumb scan-out
> buffers. In-memory data format is relevant there, but KMS only handles width,
> height, pitch, depth and bit-per-pixel information. The API doesn't care
> whether the frame buffer or the dumb scan-out buffer contains RGB or YUV data.
>
> An RFC[2] has recently been posted to the dri-devel mailing list to "add
> overlays as first class KMS objects". The proposal includes explicit overlay
> format support, and discussions have so far pushed towards reusing V4L format
> codes for the KMS API.
>
> - Video 4 Linux (V4L)
>
> The V4L API version 2 (usually called V4L2) has since the beginning included
> explicit support for data format, referred to as pixel formats.
>
> Pixel formats are identified by a 32-bit number in the form of a four
> characters code (4CC or FCC[3]). The list of FCCs defined by V4L2 is available
> in [4].
>
> A pixel format uniquely defines the layout of pixel data in memory, including
> the format type (RGB, YUV, ...), number of bits per components, components
> order and subsampling. It doesn't define the range of acceptable values for
> pixel components (such as full-range YUV vs. BT.601[5]), which is carried
> through a separate colorspace identifier[6].
>
>
> YUV support in the fbdev API
> ----------------------------
>
> Experience with the V4L2 API, both for desktop and embedded devices, and the
> KMS API, suggests that recent hardware tend to standardize on a relatively
> small number of pixel formats that don't require bitfield-level descriptions.
> A pixel format definition based on identifiers should thus fullfill the
> hardware needs for the foreseeable future.
>
> Given the overlap between the KMS, V4L2 and fbdev APIs, the need to share data
> and buffers between those subsystems, and the planned use of V4L2 FCCs in the
> KMS overlay API, I believe using V4L2 FCCs to identify fbdev formats would be
> a wise decision.
>
> To select a frame buffer YUV format, the fb_var_screeninfo structure will need
> to be extended with a format field. The fbdev API and ABI must not be broken,
> which prevents us from changing the current structure layout and replacing the
> existing format selection mechanism (through the red, green, blue and alpha
> bitfields) completely.
>
> Several solutions exist to introduce a format field in the fb_var_screeninfo
> structure.
>
> - Use the nonstd field as a format identifier. Existing drivers that use the
> nonstd field for other purposes wouldn't be able to implement the new API
> while keeping their existing API. This isn't a show stopper for drivers using
> the FB_NONSTD_HAM and FB_NONSTD_REV_PIX_IN_B bits, as they don't need to
> support any non-RGB format.
>
> Existing drivers that support YUV formats through the nonstd field would have
> to select between the current and the new API, without being able to implement
> both.
>
> - Use one of the reserved fields as a format identifier. Applications are
> supposed to zero the fb_var_screeninfo structure before passing it to the
> kernel, but experience showed that many applications forget to do so. Drivers
> would then be unable to find out whether applications request a particular
> format, or just forgot to initialize the field.
>
> - Add a new FB_NONSTD_FORMAT bit to the nonstd field, and pass the format
> through a separate field. If an application sets the FB_NONSTD_FORMAT bit in
> the nonstd field, drivers will ignore the red, green, blue, transp and
> grayscale fields, and use the format identifier to configure the frame buffer
> format. The grayscale field would then be unneeded and could be reused as a
> format identifier. Alternatively, one of the reserved fields could be used for
> that purpose.
>
> Existing drivers that support YUV formats through the nonstd field don't use
> the field's most significant bits. They could implement both their current API
> and the new API if the FB_NONSTD_FORMAT bit is stored in one of the nonstd
> MSBs.
>
> - Other solutions are possible, such as adding new ioctls. Those solutions are
> more intrusive, and require larger changes to both userspace and kernelspace
> code.
>
>
> The third solution has my preference. Comments and feedback will be
> appreciated. I will then work on a proof of concept and submit patches.
>
>
> [1] http://en.wikipedia.org/wiki/Hold_And_Modify
> [2] http://lwn.net/Articles/440192/
> [3] http://www.fourcc.org/
> [4] http://lxr.linux.no/linux+v2.6.38/include/linux/videodev2.h#L271
> [5] http://en.wikipedia.org/wiki/Rec._601
> [6] http://lxr.linux.no/linux+v2.6.38/include/linux/videodev2.h#L175
>
More information about the dri-devel
mailing list