[Mesa-dev] [RFC] New dma_buf -> EGLImage EGL extension - Final spec published!

Tom Cooksey tom.cooksey at arm.com
Mon Feb 25 03:54:40 PST 2013


Hi All,

The final spec has had enum values assigned and been published on Khronos:

http://www.khronos.org/registry/egl/extensions/EXT/EGL_EXT_image_dma_buf_import.txt

Thanks to all who've provided input.


Cheers,

Tom



> -----Original Message-----
> From: mesa-dev-bounces+tom.cooksey=arm.com at lists.freedesktop.org [mailto:mesa-dev-
> bounces+tom.cooksey=arm.com at lists.freedesktop.org] On Behalf Of Tom Cooksey
> Sent: 04 October 2012 13:10
> To: mesa-dev at lists.freedesktop.org; linaro-mm-sig at lists.linaro.org; dri-
> devel at lists.freedesktop.org; linux-media at vger.kernel.org
> Subject: [Mesa-dev] [RFC] New dma_buf -> EGLImage EGL extension - New draft!
> 
> Hi All,
> 
> After receiving a fair bit of feedback (thanks!), I've updated the
> EGL_EXT_image_dma_buf_import spec
> and expanded it to resolve a number of the issues. Please find the latest draft below and let
> me
> know any additional feedback you might have, either on the lists or by private e-mail - I
> don't mind
> which.
> 
> I think the only remaining issue now is if we need a mechanism whereby an application can
> query
> which drm_fourcc.h formats EGL supports or if just failing with EGL_BAD_MATCH when the
> application
> has use one EGL doesn't support is sufficient. Any thoughts?
> 
> 
> Cheers,
> 
> Tom
> 
> 
> --------------------8<--------------------
> 
> 
> Name
> 
>     EXT_image_dma_buf_import
> 
> Name Strings
> 
>     EGL_EXT_image_dma_buf_import
> 
> Contributors
> 
>     Jesse Barker
>     Rob Clark
>     Tom Cooksey
> 
> Contacts
> 
>     Jesse Barker (jesse 'dot' barker 'at' linaro 'dot' org)
>     Tom Cooksey (tom 'dot' cooksey 'at' arm 'dot' com)
> 
> Status
> 
>     DRAFT
> 
> Version
> 
>     Version 4, October 04, 2012
> 
> Number
> 
>     EGL Extension ???
> 
> Dependencies
> 
>     EGL 1.2 is required.
> 
>     EGL_KHR_image_base is required.
> 
>     The EGL implementation must be running on a Linux kernel supporting the
>     dma_buf buffer sharing mechanism.
> 
>     This extension is written against the wording of the EGL 1.2 Specification.
> 
> Overview
> 
>     This extension allows creating an EGLImage from a Linux dma_buf file
>     descriptor or multiple file descriptors in the case of multi-plane YUV
>     images.
> 
> New Types
> 
>     None
> 
> New Procedures and Functions
> 
>     None
> 
> New Tokens
> 
>     Accepted by the <target> parameter of eglCreateImageKHR:
> 
>         EGL_LINUX_DMA_BUF_EXT
> 
>     Accepted as an attribute in the <attrib_list> parameter of
>     eglCreateImageKHR:
> 
>         EGL_LINUX_DRM_FOURCC_EXT
>         EGL_DMA_BUF_PLANE0_FD_EXT
>         EGL_DMA_BUF_PLANE0_OFFSET_EXT
>         EGL_DMA_BUF_PLANE0_PITCH_EXT
>         EGL_DMA_BUF_PLANE1_FD_EXT
>         EGL_DMA_BUF_PLANE1_OFFSET_EXT
>         EGL_DMA_BUF_PLANE1_PITCH_EXT
>         EGL_DMA_BUF_PLANE2_FD_EXT
>         EGL_DMA_BUF_PLANE2_OFFSET_EXT
>         EGL_DMA_BUF_PLANE2_PITCH_EXT
>         EGL_YUV_COLOR_SPACE_HINT_EXT
>         EGL_SAMPLE_RANGE_HINT_EXT
>         EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT
>         EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT
> 
>     Accepted as the value for the EGL_YUV_COLOR_SPACE_HINT_EXT attribute:
> 
>         EGL_ITU_REC601_EXT
>         EGL_ITU_REC709_EXT
>         EGL_ITU_REC2020_EXT
> 
>     Accepted as the value for the EGL_SAMPLE_RANGE_HINT_EXT attribute:
> 
>         EGL_YUV_FULL_RANGE_EXT
>         EGL_YUV_NARROW_RANGE_EXT
> 
>     Accepted as the value for the EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT &
>     EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT attributes:
> 
>         EGL_YUV_CHROMA_SITING_0_EXT
>         EGL_YUV_CHROMA_SITING_0_5_EXT
> 
> 
> Additions to Chapter 2 of the EGL 1.2 Specification (EGL Operation)
> 
>     Add to section 2.5.1 "EGLImage Specification" (as defined by the
>     EGL_KHR_image_base specification), in the description of
>     eglCreateImageKHR:
> 
>    "Values accepted for <target> are listed in Table aaa, below.
> 
>       +-------------------------+--------------------------------------------+
>       |  <target>               |  Notes                                     |
>       +-------------------------+--------------------------------------------+
>       |  EGL_LINUX_DMA_BUF_EXT  |   Used for EGLImages imported from Linux   |
>       |                         |   dma_buf file descriptors                 |
>       +-------------------------+--------------------------------------------+
>        Table aaa.  Legal values for eglCreateImageKHR <target> parameter
> 
>     ...
> 
>     If <target> is EGL_LINUX_DMA_BUF_EXT, <dpy> must be a valid display, <ctx>
>     must be EGL_NO_CONTEXT, and <buffer> must be NULL, cast into the type
>     EGLClientBuffer. The details of the image is specified by the attributes
>     passed into eglCreateImageKHR. Required attributes and their values are as
>     follows:
> 
>         * EGL_WIDTH & EGL_HEIGHT: The logical dimensions of the buffer in pixels
> 
>         * EGL_LINUX_DRM_FOURCC_EXT: The pixel format of the buffer, as specified
>           by drm_fourcc.h and used as the pixel_format parameter of the
>           drm_mode_fb_cmd2 ioctl.
> 
>         * EGL_DMA_BUF_PLANE0_FD_EXT: The dma_buf file descriptor of plane 0 of
>           the image.
> 
>         * EGL_DMA_BUF_PLANE0_OFFSET_EXT: The offset from the start of the
>           dma_buf of the first sample in plane 0, in bytes.
> 
>         * EGL_DMA_BUF_PLANE0_PITCH_EXT: The number of bytes between the start of
>           subsequent rows of samples in plane 0. May have special meaning for
>           non-linear formats.
> 
>     For images in an RGB color-space or those using a single-plane YUV format,
>     only the first plane's file descriptor, offset & pitch should be specified.
>     For semi-planar YUV formats, the chroma samples are stored in plane 1 and
>     for fully planar formats, U-samples are stored in plane 1 and V-samples are
>     stored in plane 2. Planes 1 & 2 are specified by the following attributes,
>     which have the same meanings as defined above for plane 0:
> 
>         * EGL_DMA_BUF_PLANE1_FD_EXT
>         * EGL_DMA_BUF_PLANE1_OFFSET_EXT
>         * EGL_DMA_BUF_PLANE1_PITCH_EXT
>         * EGL_DMA_BUF_PLANE2_FD_EXT
>         * EGL_DMA_BUF_PLANE2_OFFSET_EXT
>         * EGL_DMA_BUF_PLANE2_PITCH_EXT
> 
>     In addition to the above required attributes, the application may also
>     provide hints as to how the data should be interpreted by the GL. If any of
>     these hints are not specified, the GL will guess based on the pixel format
>     passed as the EGL_LINUX_DRM_FOURCC_EXT attribute or may fall-back to some
>     default value. Not all GLs will be able to support all combinations of
>     these hints and are free to use whatever settings they choose to achieve
>     the closest possible match.
> 
>         * EGL_YUV_COLOR_SPACE_HINT_EXT: The color-space the data is in. Only
>           relevant for images in a YUV format, ignored when specified for an
>           image in an RGB format. Accepted values are:
>           EGL_ITU_REC601_EXT, EGL_ITU_REC709_EXT & EGL_ITU_REC2020_EXT.
> 
>         * EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT &
>           EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT: Where chroma samples are
>           sited relative to luma samples when the image is in a sub-sampled
>           format. When the image is not using chroma sub-sampling, the luma and
>           chroma samples are assumed to be co-sited. Siting is split into the
>           vertical and horizontal and is in a fixed range. A siting of zero
>           means the first luma sample is taken from the same position in that
>           dimension as the chroma sample. This is best illustrated in the
>           diagram below:
> 
>                  (0.5, 0.5)        (0.0, 0.5)        (0.0, 0.0)
>                 +   +   +   +     +   +   +   +     *   +   *   +
>                   x       x       x       x
>                 +   +   +   +     +   +   +   +     +   +   +   +
> 
>                 +   +   +   +     +   +   +   +     *   +   *   +
>                   x       x       x       x
>                 +   +   +   +     +   +   +   +     +   +   +   +
> 
>             Luma samples (+), Chroma samples (x) Chrome & Luma samples (*)
> 
>           Note this attribute is ignored for RGB images and non sub-sampled
>           YUV images. Accepted values are: EGL_YUV_CHROMA_SITING_0_EXT (0.0)
>           & EGL_YUV_CHROMA_SITING_0_5_EXT (0.5)
> 
>         * EGL_SAMPLE_RANGE_HINT_EXT: The numerical range of samples. Only
>           relevant for images in a YUV format, ignored when specified for
>           images in an RGB format. Accepted values are: EGL_YUV_FULL_RANGE_EXT
>           (0-256) & EGL_YUV_NARROW_RANGE_EXT (16-235).
> 
> 
>     If eglCreateImageKHR is successful for a EGL_LINUX_DMA_BUF_EXT target, the
>     EGL takes ownership of the file descriptor and is responsible for closing
>     it, which it may do at any time while the EGLDisplay is initialized."
> 
> 
>     Add to the list of error conditions for eglCreateImageKHR:
> 
>       "* If <target> is EGL_LINUX_DMA_BUF_EXT and <buffer> is not NULL, the
>          error EGL_BAD_PARAMETER is generated.
> 
>        * If <target> is EGL_LINUX_DMA_BUF_EXT, and the list of attributes is
>          incomplete, EGL_BAD_PARAMETER is generated.
> 
>        * If <target> is EGL_LINUX_DMA_BUF_EXT, and the EGL_LINUX_DRM_FOURCC_EXT
>          attribute is set to a format not supported by the EGL, EGL_BAD_MATCH
>          is generated.
> 
>        * If <target> is EGL_LINUX_DMA_BUF_EXT, and the EGL_LINUX_DRM_FOURCC_EXT
>          attribute indicates a single-plane format, EGL_BAD_ATTRIBUTE is
>          generated if any of the EGL_DMA_BUF_PLANE1_* or EGL_DMA_BUF_PLANE2_*
>          attributes are specified.
> 
>        * If <target> is EGL_LINUX_DMA_BUF_EXT and the value specified for
>          EGL_YUV_COLOR_SPACE_HINT_EXT is not EGL_ITU_REC601_EXT,
>          EGL_ITU_REC709_EXT or EGL_ITU_REC2020_EXT, EGL_BAD_ATTRIBUTE is
>          generated.
> 
>        * If <target> is EGL_LINUX_DMA_BUF_EXT and the value specified for
>          EGL_SAMPLE_RANGE_HINT_EXT is not EGL_YUV_FULL_RANGE_EXT or
>          EGL_YUV_NARROW_RANGE_EXT, EGL_BAD_ATTRIBUTE is generated.
> 
>        * If <target> is EGL_LINUX_DMA_BUF_EXT and the value specified for
>          EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT or
>          EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT is not
>          EGL_YUV_CHROMA_SITING_0_EXT or EGL_YUV_CHROMA_SITING_0_5_EXT,
>          EGL_BAD_ATTRIBUTE is generated.
> 
>        * If <target> is EGL_LINUX_DMA_BUF_EXT and one or more of the values
>          specified for a plane's pitch or offset isn't supported by EGL,
>          EGL_BAD_ACCESS is generated.
> 
>        * If <target> is EGL_LINUX_DMA_BUF_EXT and eglCreateImageKHR fails,
>          EGL does not retain ownership of the file descriptor and it is the
>          responsibility of the application to close it."
> 
> 
> Issues
> 
>     1. Should this be a KHR or EXT extension?
> 
>     ANSWER: EXT. Khronos EGL working group not keen on this extension as it is
>     seen as contradicting the EGLStream direction the specification is going in.
>     The working group recommends creating additional specs to allow an EGLStream
>     producer/consumer connected to v4l2/DRM or any other Linux interface.
> 
>     2. Should this be a generic any platform extension, or a Linux-only
>     extension which explicitly states the handles are dma_buf fds?
> 
>     ANSWER: There's currently no intention to port this extension to any OS not
>     based on the Linux kernel. Consequently, this spec can be explicitly written
>     against Linux and the dma_buf API.
> 
>     3. Does ownership of the file descriptor pass to the EGL library?
> 
>     ANSWER: If eglCreateImageKHR is successful, EGL assumes ownership of the
>     file descriptors and is responsible for closing them.
> 
>     4. How are the different YUV color spaces handled (BT.709/BT.601)?
> 
>     ANSWER: The pixel formats defined in drm_fourcc.h only specify how the data
>     is laid out in memory. It does not define how that data should be
>     interpreted. Added a new EGL_YUV_COLOR_SPACE_HINT_EXT attribute to allow the
>     application to specify which color space the data is in to allow the GL to
>     choose an appropriate set of co-efficients if it needs to convert that data
>     to RGB for example.
> 
>     5. What chroma-siting is used for sub-sampled YUV formats?
> 
>     ANSWER: The chroma siting is not specified by either the v4l2 or DRM APIs.
>     This is similar to the color-space issue (4) in that the chroma siting
>     doesn't affect how the data is stored in memory. However, the GL will need
>     to know the siting in order to filter the image correctly. While the visual
>     impact of getting the siting wrong is minor, provision should be made to
>     allow an application to specify the siting if desired. Added additional
>     EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT &
>     EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT attributes to allow the siting to
>     be specified using a set of pre-defined values (0 or 0.5).
> 
>     6. How can an application query which formats the EGL implementation
>     supports?
> 
>     PROPOSAL: Don't provide a query mechanism but instead add an error condition
>     that EGL_BAD_MATCH is raised if the EGL implementation doesn't support that
>     particular format.
> 
>     7. Which image formats should be supported and how is format specified?
> 
>     Seem to be two options 1) specify a new enum in this specification and
>     enumerate all possible formats. 2) Use an existing enum already in Linux,
>     either v4l2_mbus_pixelcode and/or those formats listed in drm_fourcc.h?
> 
>     ANSWER: Go for option 2) and just use values defined in drm_fourcc.h.
> 
>     8. How can AYUV images be handled?
> 
>     ANSWER: At least on fourcc.org and in drm_fourcc.h, there only seems to be
>     a single AYUV format and that is a packed format, so everything, including
>     the alpha component would be in the first plane.
> 
>     9. How can you import interlaced images?
> 
>     ANSWER: Interlaced frames are usually stored with the top & bottom fields
>     interleaved in a single buffer. As the fields would need to be displayed as
>     at different times, the application would create two EGLImages from the same
>     buffer, one for the top field and another for the bottom. Both EGLImages
>     would set the pitch to 2x the buffer width and the second EGLImage would use
>     a suitable offset to indicate it started on the second line of the buffer.
>     This should work regardless of whether the data is packed in a single plane,
>     semi-planar or multi-planar.
> 
>     If each interlaced field is stored in a separate buffer then it should be
>     trivial to create two EGLImages, one for each field's buffer.
> 
>     10. How are semi-planar/planar formats handled that have a different
>     width/height for Y' and CbCr such as YUV420?
> 
>     ANSWER: The spec says EGL_WIDTH & EGL_HEIGHT specify the *logical* width and
>     height of the buffer in pixels. For pixel formats with sub-sampled Chroma
>     values, it should be trivial for the EGL implementation to calculate the
>     width/height of the Chroma sample buffers using the logical width & height
>     and by inspecting the pixel format passed as the EGL_LINUX_DRM_FOURCC_EXT
>     attribute. I.e. If the pixel format says it's YUV420, the Chroma buffer's
>     width = EGL_WIDTH/2 & height =EGL_HEIGHT/2.
> 
>     11. How are Bayer formats handled?
> 
>     ANSWER: As of Linux 2.6.34, drm_fourcc.h does not include any Bayer formats.
>     However, future kernel versions may add such formats in which case they
>     would be handled in the same way as any other format.
> 
>     12. Should the spec support buffers which have samples in a "narrow range"?
> 
>     Content sampled from older analogue sources typically don't use the full
>     (0-256) range of the data type storing the sample and instead use a narrow
>     (16-235) range to allow some headroom & toeroom in the signals to avoid
>     clipping signals which overshoot slightly during processing. This is
>     sometimes known as signals using "studio swing".
> 
>     ANSWER: Add a new attribute to define if the samples use a narrow 16-235
>     range or the full 0-256 range.
> 
>     13. Specifying the color space and range seems cumbersome, why not just
>     allow the application to specify the full YUV->RGB color conversion matrix?
> 
>     ANSWER: Some hardware may not be able to use an arbitrary conversion matrix
>     and needs to select an appropriate pre-defined matrix based on the color
>     space and the sample range.
> 
>     14. How do you handle EGL implementations which have restrictions on pitch
>     and/or offset?
> 
>     ANSWER: Buffers being imported using dma_buf pretty much have to be
>     allocated by a kernel-space driver. As such, it is expected that a system
>     integrator would make sure all devices which allocate buffers suitable for
>     exporting make sure they use a pitch supported by all possible importers.
>     However, it is still possible eglCreateImageKHR can fail due to an
>     unsupported pitch. Added a new error to the list indicating this.
> 
>     15. Should this specification also describe how to export an existing
>     EGLImage as a dma_buf file descriptor?
> 
>     ANSWER: No. Importing and exporting buffers are two separate operations and
>     importing an existing dma_buf fd into an EGLImage is useful functionality in
>     itself. Agree that exporting an EGLImage as a dma_buf fd is useful, E.g. it
>     could be used by an OpenMAX IL implementation's OMX_UseEGLImage function to
>     give access to the buffer backing an EGLImage to video hardware. However,
>     exporting can be split into a separate extension specification.
> 
> 
> Revision History
> 
> #4 (Tom Cooksey, October 04, 2012)
>    - Fixed issue numbering!
>    - Added issues 8 - 15.
>    - Promoted proposal for Issue 3 to be the answer.
>    - Added an additional attribute to allow an application to specify the color
>      space as a hint which should address issue 4.
>    - Added an additional attribute to allow an application to specify the chroma
>      siting as a hint which should address issue 5.
>    - Added an additional attribute to allow an application to specify the sample
>      range as a hint which should address the new issue 12.
>    - Added language to end of error section clarifying who owns the fd passed
>      to eglCreateImageKHR if an error is generated.
> 
> #3 (Tom Cooksey, August 16, 2012)
>    - Changed name from EGL_EXT_image_external and re-written language to
>      explicitly state this for use with Linux & dma_buf.
>    - Added a list of issues, including some still open ones.
> 
> #2 (Jesse Barker, May 30, 2012)
>    - Revision to split eglCreateImageKHR functionality from export
>      Functionality.
>    - Update definition of EGLNativeBufferType to be a struct containing a list
>      of handles to support multi-buffer/multi-planar formats.
> 
> #1 (Jesse Barker, March 20, 2012)
>    - Initial draft.
> 
> 
> 
> 
> _______________________________________________
> mesa-dev mailing list
> mesa-dev at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/mesa-dev






More information about the mesa-dev mailing list