[Mesa-dev] [RFC] New dma_buf -> EGLImage EGL extension

Thu Aug 30 07:00:00 PDT 2012

Hi All,

Over the last few months I've been working on & off with a few people from
Linaro on a new EGL extension. The extension allows constructing an EGLImage
from a (set of) dma_buf file descriptors, including support for multi-plane
YUV. I envisage the primary use-case of this extension to be importing video
frames from v4l2 into the EGL/GLES graphics driver to texture from.
Originally the intent was to develop this as a Khronos-ratified extension.
However, this is a little too platform-specific to be an officially
sanctioned Khronos extension. It also goes against the general "EGLStream"
direction the EGL working group is going in. As such, the general feeling
was to make this an EXT "multi-vendor" extension with no official stamp of
approval from Khronos. As this is no-longer intended to be a Khronos
extension, I've re-written it to be a lot more Linux & dma_buf specific. It
also allows me to circulate the extension more widely (I.e. To those outside
Khronos membership).

ARM are implementing this extension for at least our Mali-T6xx driver and
likely earlier drivers too. I am sending this e-mail to solicit feedback,
both from other vendors who might implement this extension (Mesa3D?) and
from potential users of the extension. However, any feedback is welcome.
Please find the extension text as it currently stands below. There several
open issues which I've proposed solutions for, but I'm not really happy with
those proposals and hoped others could chip-in with better ideas. There are
likely other issues I've not thought about which also need to be added and
addressed.

Once there's a general consensus or if no-one's interested, I'll update the
spec, move it out of Draft status and get it added to the Khronos registry,
which includes assigning values for the new symbols.

Cheers,

Tom

---------8<---------

Name

    EXT_image_dma_buf_import

Name Strings

    EGL_EXT_image_dma_buf_import

Contributors

    Jesse Barker
    Rob Clark
    Tom Cooksey

Contacts

    Jesse Barker (jesse 'dot' barker 'at' linaro 'dot' org)
    Tom Cooksey (tom 'dot' cooksey 'at' arm 'dot' com)

Status

    DRAFT

Version

    Version 3, August 16, 2012

Number

    EGL Extension ???

Dependencies

    EGL 1.2 is required.

    EGL_KHR_image_base is required.

    The EGL implementation must be running on a Linux kernel supporting the
    dma_buf buffer sharing mechanism.

    This extension is written against the wording of the EGL 1.2
Specification.

Overview

    This extension allows creating an EGLImage from a Linux dma_buf file
    descriptor or multiple file descriptors in the case of multi-plane YUV
    images.

New Types

    None

New Procedures and Functions

    None

New Tokens

    Accepted by the <target> parameter of eglCreateImageKHR:

        EGL_LINUX_DMA_BUF_EXT

    Accepted as an attribute in the <attrib_list> parameter of
    eglCreateImageKHR:

        EGL_LINUX_DRM_FOURCC_EXT
        EGL_DMA_BUF_PLANE0_FD_EXT
        EGL_DMA_BUF_PLANE0_OFFSET_EXT
        EGL_DMA_BUF_PLANE0_PITCH_EXT
        EGL_DMA_BUF_PLANE1_FD_EXT
        EGL_DMA_BUF_PLANE1_OFFSET_EXT
        EGL_DMA_BUF_PLANE1_PITCH_EXT
        EGL_DMA_BUF_PLANE2_FD_EXT
        EGL_DMA_BUF_PLANE2_OFFSET_EXT
        EGL_DMA_BUF_PLANE2_PITCH_EXT

Additions to Chapter 2 of the EGL 1.2 Specification (EGL Operation)

    Add to section 2.5.1 "EGLImage Specification" (as defined by the
    EGL_KHR_image_base specification), in the description of
    eglCreateImageKHR:

   "Values accepted for <target> are listed in Table aaa, below.

+-------------------------+--------------------------------------------+
      |  <target>               |  Notes
|

+-------------------------+--------------------------------------------+
      |  EGL_LINUX_DMA_BUF_EXT  |   Used for EGLImages imported from Linux
|
      |                         |   dma_buf file descriptors
|

+-------------------------+--------------------------------------------+
       Table aaa.  Legal values for eglCreateImageKHR <target> parameter

    ...

    If <target> is EGL_LINUX_DMA_BUF_EXT, <dpy> must be a valid display,
<ctx>
    must be EGL_NO_CONTEXT, and <buffer> must be NULL, cast into the type
    EGLClientBuffer. The details of the image is specified by the attributes
    passed into eglCreateImageKHR. Required attributes and their values are
as
    follows:

        * EGL_WIDTH & EGL_HEIGHT: The logical dimensions of the buffer in
pixels

        * EGL_LINUX_DRM_FOURCC_EXT: The pixel format of the buffer, as
specified
          by drm_fourcc.h and used as the pixel_format parameter of the
          drm_mode_fb_cmd2 ioctl.

        * EGL_DMA_BUF_PLANE0_FD_EXT: The dma_buf file descriptor of plane 0
of
          the image.

        * EGL_DMA_BUF_PLANE0_OFFSET_EXT: The offset from the start of the
          dma_buf of the first sample in plane 0, in bytes.

        * EGL_DMA_BUF_PLANE0_PITCH_EXT: The number of bytes between the
start of
          subsequent rows of samples in plane 0. May have special meaning
for
          non-linear formats.

    For images in an RGB color-space or those using a single-plane YUV
format,
    only the first plane's file descriptor, offset & pitch should be
specified.
    For semi-planar YUV formats, the chroma samples are stored in plane 1
and
    for fully planar formats, U-samples are stored in plane 1 and V-samples
are
    stored in plane 2. Planes 1 & 2 are specified by the following
attributes,
    which have the same meanings as defined above for plane 0:

        * EGL_DMA_BUF_PLANE1_FD_EXT
        * EGL_DMA_BUF_PLANE1_OFFSET_EXT
        * EGL_DMA_BUF_PLANE1_PITCH_EXT
        * EGL_DMA_BUF_PLANE2_FD_EXT
        * EGL_DMA_BUF_PLANE2_OFFSET_EXT
        * EGL_DMA_BUF_PLANE2_PITCH_EXT

    If eglCreateImageKHR is successful for a EGL_LINUX_DMA_BUF_EXT target,
the
    EGL takes ownership of the file descriptor and is responsible for
closing
    it, which it may do at any time while the EGLDisplay is initialized."

    Add to the list of error conditions for eglCreateImageKHR:

      "* If <target> is EGL_LINUX_DMA_BUF_EXT and <buffer> is not NULL, the
         error EGL_BAD_PARAMETER is generated.

       * If <target> is EGL_LINUX_DMA_BUF_EXT, and the list of attributes is
         incomplete, EGL_BAD_PARAMETER is generated.

       * If <target> is EGL_LINUX_DMA_BUF_EXT, and the
EGL_LINUX_DRM_FOURCC_EXT
         attribute is set to a format not supported by the EGL,
EGL_BAD_MATCH
         is generated.

       * If <target> is EGL_LINUX_DMA_BUF_EXT, and the
EGL_LINUX_DRM_FOURCC_EXT
         attribute indicates a single-plane format, EGL_BAD_ATTRIBUTE is
         generated if any of the EGL_DMA_BUF_PLANE1_* or
EGL_DMA_BUF_PLANE2_*
         attributes are specified.

Issues

    1. Should this be a KHR or EXT extension?

    ANSWER: EXT. Khronos EGL working group not keen on this extension as it
is
    seen as contradicting the EGLStream direction the specification is going
in.
    The working group recommends creating additional specs to allow an
EGLStream
    producer/consumer connected to v4l2/DRM or any other Linux interface.

    2. Should this be a generic any platform extension, or a Linux-only
    extension which explicitly states the handles are dma_buf fds?

    ANSWER: There's currently no intention to port this extension to any OS
not
    based on the Linux kernel. Consequently, this spec can be explicitly
written
    against Linux and the dma_buf API.

    3. Does ownership of the file descriptor pass to the EGL library?

    PROPOSAL: If eglCreateImageKHR is successful, EGL assumes ownership of
the
    file descriptors and is responsible for closing them.

    4. How are the different YUV color spaces handled (BT.709/BT.601)?

    Open issue, still TBD. Doesn't seem to be specified by either the v4l2
or
    DRM APIs. PROPOSAL: Undefined and implementation/format dependent.

    5. What chroma-siting is used for sub-sampled YUV formats?

    Open issue, still TBD. Doesn't seem to be specified by either the v4l2
or
    DRM APIs. PROPOSAL: Undefined and implementation/format dependent.

    5. How can an application query which formats the EGL implementation
    supports?

    PROPOSAL: Don't provide a query mechanism but instead add an error
condition
    that EGL_BAD_MATCH is raised if the EGL implementation doesn't support
that
    particular format.

    5. Which image formats should be supported and how is format specified?

    Open issue, still TBD. Seem to be two options 1) specify a new enum in
this
    specification and enumerate all possible formats. 2) Use an existing
enum
    already in Linux, either v4l2_mbus_pixelcode and/or those formats listed
    in drm_fourcc.h?
    PROPOSAL: Go for option 2) and just use values defined in drm_fourcc.h.

Revision History

#3 (Tom Cooksey, August 16, 2012)
   - Changed name from EGL_EXT_image_external and re-written language to
     explicitly state this for use with Linux & dma_buf.
   - Added a list of issues, including some still open ones.

#2 (Jesse Barker, May 30, 2012)
   - Revision to split eglCreateImageKHR functionality from export
     Functionality.
   - Update definition of EGLNativeBufferType to be a struct containing a
list
     of handles to support multi-buffer/multi-planar formats.

#1 (Jesse Barker, March 20, 2012)
   - Initial draft.