[Mesa-dev] [RFC] New dma_buf -> EGLImage EGL extension - Final spec published!
Tom Cooksey
tom.cooksey at arm.com
Mon Feb 25 03:54:40 PST 2013
Hi All,
The final spec has had enum values assigned and been published on Khronos:
http://www.khronos.org/registry/egl/extensions/EXT/EGL_EXT_image_dma_buf_import.txt
Thanks to all who've provided input.
Cheers,
Tom
> -----Original Message-----
> From: mesa-dev-bounces+tom.cooksey=arm.com at lists.freedesktop.org [mailto:mesa-dev-
> bounces+tom.cooksey=arm.com at lists.freedesktop.org] On Behalf Of Tom Cooksey
> Sent: 04 October 2012 13:10
> To: mesa-dev at lists.freedesktop.org; linaro-mm-sig at lists.linaro.org; dri-
> devel at lists.freedesktop.org; linux-media at vger.kernel.org
> Subject: [Mesa-dev] [RFC] New dma_buf -> EGLImage EGL extension - New draft!
>
> Hi All,
>
> After receiving a fair bit of feedback (thanks!), I've updated the
> EGL_EXT_image_dma_buf_import spec
> and expanded it to resolve a number of the issues. Please find the latest draft below and let
> me
> know any additional feedback you might have, either on the lists or by private e-mail - I
> don't mind
> which.
>
> I think the only remaining issue now is if we need a mechanism whereby an application can
> query
> which drm_fourcc.h formats EGL supports or if just failing with EGL_BAD_MATCH when the
> application
> has use one EGL doesn't support is sufficient. Any thoughts?
>
>
> Cheers,
>
> Tom
>
>
> --------------------8<--------------------
>
>
> Name
>
> EXT_image_dma_buf_import
>
> Name Strings
>
> EGL_EXT_image_dma_buf_import
>
> Contributors
>
> Jesse Barker
> Rob Clark
> Tom Cooksey
>
> Contacts
>
> Jesse Barker (jesse 'dot' barker 'at' linaro 'dot' org)
> Tom Cooksey (tom 'dot' cooksey 'at' arm 'dot' com)
>
> Status
>
> DRAFT
>
> Version
>
> Version 4, October 04, 2012
>
> Number
>
> EGL Extension ???
>
> Dependencies
>
> EGL 1.2 is required.
>
> EGL_KHR_image_base is required.
>
> The EGL implementation must be running on a Linux kernel supporting the
> dma_buf buffer sharing mechanism.
>
> This extension is written against the wording of the EGL 1.2 Specification.
>
> Overview
>
> This extension allows creating an EGLImage from a Linux dma_buf file
> descriptor or multiple file descriptors in the case of multi-plane YUV
> images.
>
> New Types
>
> None
>
> New Procedures and Functions
>
> None
>
> New Tokens
>
> Accepted by the <target> parameter of eglCreateImageKHR:
>
> EGL_LINUX_DMA_BUF_EXT
>
> Accepted as an attribute in the <attrib_list> parameter of
> eglCreateImageKHR:
>
> EGL_LINUX_DRM_FOURCC_EXT
> EGL_DMA_BUF_PLANE0_FD_EXT
> EGL_DMA_BUF_PLANE0_OFFSET_EXT
> EGL_DMA_BUF_PLANE0_PITCH_EXT
> EGL_DMA_BUF_PLANE1_FD_EXT
> EGL_DMA_BUF_PLANE1_OFFSET_EXT
> EGL_DMA_BUF_PLANE1_PITCH_EXT
> EGL_DMA_BUF_PLANE2_FD_EXT
> EGL_DMA_BUF_PLANE2_OFFSET_EXT
> EGL_DMA_BUF_PLANE2_PITCH_EXT
> EGL_YUV_COLOR_SPACE_HINT_EXT
> EGL_SAMPLE_RANGE_HINT_EXT
> EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT
> EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT
>
> Accepted as the value for the EGL_YUV_COLOR_SPACE_HINT_EXT attribute:
>
> EGL_ITU_REC601_EXT
> EGL_ITU_REC709_EXT
> EGL_ITU_REC2020_EXT
>
> Accepted as the value for the EGL_SAMPLE_RANGE_HINT_EXT attribute:
>
> EGL_YUV_FULL_RANGE_EXT
> EGL_YUV_NARROW_RANGE_EXT
>
> Accepted as the value for the EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT &
> EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT attributes:
>
> EGL_YUV_CHROMA_SITING_0_EXT
> EGL_YUV_CHROMA_SITING_0_5_EXT
>
>
> Additions to Chapter 2 of the EGL 1.2 Specification (EGL Operation)
>
> Add to section 2.5.1 "EGLImage Specification" (as defined by the
> EGL_KHR_image_base specification), in the description of
> eglCreateImageKHR:
>
> "Values accepted for <target> are listed in Table aaa, below.
>
> +-------------------------+--------------------------------------------+
> | <target> | Notes |
> +-------------------------+--------------------------------------------+
> | EGL_LINUX_DMA_BUF_EXT | Used for EGLImages imported from Linux |
> | | dma_buf file descriptors |
> +-------------------------+--------------------------------------------+
> Table aaa. Legal values for eglCreateImageKHR <target> parameter
>
> ...
>
> If <target> is EGL_LINUX_DMA_BUF_EXT, <dpy> must be a valid display, <ctx>
> must be EGL_NO_CONTEXT, and <buffer> must be NULL, cast into the type
> EGLClientBuffer. The details of the image is specified by the attributes
> passed into eglCreateImageKHR. Required attributes and their values are as
> follows:
>
> * EGL_WIDTH & EGL_HEIGHT: The logical dimensions of the buffer in pixels
>
> * EGL_LINUX_DRM_FOURCC_EXT: The pixel format of the buffer, as specified
> by drm_fourcc.h and used as the pixel_format parameter of the
> drm_mode_fb_cmd2 ioctl.
>
> * EGL_DMA_BUF_PLANE0_FD_EXT: The dma_buf file descriptor of plane 0 of
> the image.
>
> * EGL_DMA_BUF_PLANE0_OFFSET_EXT: The offset from the start of the
> dma_buf of the first sample in plane 0, in bytes.
>
> * EGL_DMA_BUF_PLANE0_PITCH_EXT: The number of bytes between the start of
> subsequent rows of samples in plane 0. May have special meaning for
> non-linear formats.
>
> For images in an RGB color-space or those using a single-plane YUV format,
> only the first plane's file descriptor, offset & pitch should be specified.
> For semi-planar YUV formats, the chroma samples are stored in plane 1 and
> for fully planar formats, U-samples are stored in plane 1 and V-samples are
> stored in plane 2. Planes 1 & 2 are specified by the following attributes,
> which have the same meanings as defined above for plane 0:
>
> * EGL_DMA_BUF_PLANE1_FD_EXT
> * EGL_DMA_BUF_PLANE1_OFFSET_EXT
> * EGL_DMA_BUF_PLANE1_PITCH_EXT
> * EGL_DMA_BUF_PLANE2_FD_EXT
> * EGL_DMA_BUF_PLANE2_OFFSET_EXT
> * EGL_DMA_BUF_PLANE2_PITCH_EXT
>
> In addition to the above required attributes, the application may also
> provide hints as to how the data should be interpreted by the GL. If any of
> these hints are not specified, the GL will guess based on the pixel format
> passed as the EGL_LINUX_DRM_FOURCC_EXT attribute or may fall-back to some
> default value. Not all GLs will be able to support all combinations of
> these hints and are free to use whatever settings they choose to achieve
> the closest possible match.
>
> * EGL_YUV_COLOR_SPACE_HINT_EXT: The color-space the data is in. Only
> relevant for images in a YUV format, ignored when specified for an
> image in an RGB format. Accepted values are:
> EGL_ITU_REC601_EXT, EGL_ITU_REC709_EXT & EGL_ITU_REC2020_EXT.
>
> * EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT &
> EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT: Where chroma samples are
> sited relative to luma samples when the image is in a sub-sampled
> format. When the image is not using chroma sub-sampling, the luma and
> chroma samples are assumed to be co-sited. Siting is split into the
> vertical and horizontal and is in a fixed range. A siting of zero
> means the first luma sample is taken from the same position in that
> dimension as the chroma sample. This is best illustrated in the
> diagram below:
>
> (0.5, 0.5) (0.0, 0.5) (0.0, 0.0)
> + + + + + + + + * + * +
> x x x x
> + + + + + + + + + + + +
>
> + + + + + + + + * + * +
> x x x x
> + + + + + + + + + + + +
>
> Luma samples (+), Chroma samples (x) Chrome & Luma samples (*)
>
> Note this attribute is ignored for RGB images and non sub-sampled
> YUV images. Accepted values are: EGL_YUV_CHROMA_SITING_0_EXT (0.0)
> & EGL_YUV_CHROMA_SITING_0_5_EXT (0.5)
>
> * EGL_SAMPLE_RANGE_HINT_EXT: The numerical range of samples. Only
> relevant for images in a YUV format, ignored when specified for
> images in an RGB format. Accepted values are: EGL_YUV_FULL_RANGE_EXT
> (0-256) & EGL_YUV_NARROW_RANGE_EXT (16-235).
>
>
> If eglCreateImageKHR is successful for a EGL_LINUX_DMA_BUF_EXT target, the
> EGL takes ownership of the file descriptor and is responsible for closing
> it, which it may do at any time while the EGLDisplay is initialized."
>
>
> Add to the list of error conditions for eglCreateImageKHR:
>
> "* If <target> is EGL_LINUX_DMA_BUF_EXT and <buffer> is not NULL, the
> error EGL_BAD_PARAMETER is generated.
>
> * If <target> is EGL_LINUX_DMA_BUF_EXT, and the list of attributes is
> incomplete, EGL_BAD_PARAMETER is generated.
>
> * If <target> is EGL_LINUX_DMA_BUF_EXT, and the EGL_LINUX_DRM_FOURCC_EXT
> attribute is set to a format not supported by the EGL, EGL_BAD_MATCH
> is generated.
>
> * If <target> is EGL_LINUX_DMA_BUF_EXT, and the EGL_LINUX_DRM_FOURCC_EXT
> attribute indicates a single-plane format, EGL_BAD_ATTRIBUTE is
> generated if any of the EGL_DMA_BUF_PLANE1_* or EGL_DMA_BUF_PLANE2_*
> attributes are specified.
>
> * If <target> is EGL_LINUX_DMA_BUF_EXT and the value specified for
> EGL_YUV_COLOR_SPACE_HINT_EXT is not EGL_ITU_REC601_EXT,
> EGL_ITU_REC709_EXT or EGL_ITU_REC2020_EXT, EGL_BAD_ATTRIBUTE is
> generated.
>
> * If <target> is EGL_LINUX_DMA_BUF_EXT and the value specified for
> EGL_SAMPLE_RANGE_HINT_EXT is not EGL_YUV_FULL_RANGE_EXT or
> EGL_YUV_NARROW_RANGE_EXT, EGL_BAD_ATTRIBUTE is generated.
>
> * If <target> is EGL_LINUX_DMA_BUF_EXT and the value specified for
> EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT or
> EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT is not
> EGL_YUV_CHROMA_SITING_0_EXT or EGL_YUV_CHROMA_SITING_0_5_EXT,
> EGL_BAD_ATTRIBUTE is generated.
>
> * If <target> is EGL_LINUX_DMA_BUF_EXT and one or more of the values
> specified for a plane's pitch or offset isn't supported by EGL,
> EGL_BAD_ACCESS is generated.
>
> * If <target> is EGL_LINUX_DMA_BUF_EXT and eglCreateImageKHR fails,
> EGL does not retain ownership of the file descriptor and it is the
> responsibility of the application to close it."
>
>
> Issues
>
> 1. Should this be a KHR or EXT extension?
>
> ANSWER: EXT. Khronos EGL working group not keen on this extension as it is
> seen as contradicting the EGLStream direction the specification is going in.
> The working group recommends creating additional specs to allow an EGLStream
> producer/consumer connected to v4l2/DRM or any other Linux interface.
>
> 2. Should this be a generic any platform extension, or a Linux-only
> extension which explicitly states the handles are dma_buf fds?
>
> ANSWER: There's currently no intention to port this extension to any OS not
> based on the Linux kernel. Consequently, this spec can be explicitly written
> against Linux and the dma_buf API.
>
> 3. Does ownership of the file descriptor pass to the EGL library?
>
> ANSWER: If eglCreateImageKHR is successful, EGL assumes ownership of the
> file descriptors and is responsible for closing them.
>
> 4. How are the different YUV color spaces handled (BT.709/BT.601)?
>
> ANSWER: The pixel formats defined in drm_fourcc.h only specify how the data
> is laid out in memory. It does not define how that data should be
> interpreted. Added a new EGL_YUV_COLOR_SPACE_HINT_EXT attribute to allow the
> application to specify which color space the data is in to allow the GL to
> choose an appropriate set of co-efficients if it needs to convert that data
> to RGB for example.
>
> 5. What chroma-siting is used for sub-sampled YUV formats?
>
> ANSWER: The chroma siting is not specified by either the v4l2 or DRM APIs.
> This is similar to the color-space issue (4) in that the chroma siting
> doesn't affect how the data is stored in memory. However, the GL will need
> to know the siting in order to filter the image correctly. While the visual
> impact of getting the siting wrong is minor, provision should be made to
> allow an application to specify the siting if desired. Added additional
> EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT &
> EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT attributes to allow the siting to
> be specified using a set of pre-defined values (0 or 0.5).
>
> 6. How can an application query which formats the EGL implementation
> supports?
>
> PROPOSAL: Don't provide a query mechanism but instead add an error condition
> that EGL_BAD_MATCH is raised if the EGL implementation doesn't support that
> particular format.
>
> 7. Which image formats should be supported and how is format specified?
>
> Seem to be two options 1) specify a new enum in this specification and
> enumerate all possible formats. 2) Use an existing enum already in Linux,
> either v4l2_mbus_pixelcode and/or those formats listed in drm_fourcc.h?
>
> ANSWER: Go for option 2) and just use values defined in drm_fourcc.h.
>
> 8. How can AYUV images be handled?
>
> ANSWER: At least on fourcc.org and in drm_fourcc.h, there only seems to be
> a single AYUV format and that is a packed format, so everything, including
> the alpha component would be in the first plane.
>
> 9. How can you import interlaced images?
>
> ANSWER: Interlaced frames are usually stored with the top & bottom fields
> interleaved in a single buffer. As the fields would need to be displayed as
> at different times, the application would create two EGLImages from the same
> buffer, one for the top field and another for the bottom. Both EGLImages
> would set the pitch to 2x the buffer width and the second EGLImage would use
> a suitable offset to indicate it started on the second line of the buffer.
> This should work regardless of whether the data is packed in a single plane,
> semi-planar or multi-planar.
>
> If each interlaced field is stored in a separate buffer then it should be
> trivial to create two EGLImages, one for each field's buffer.
>
> 10. How are semi-planar/planar formats handled that have a different
> width/height for Y' and CbCr such as YUV420?
>
> ANSWER: The spec says EGL_WIDTH & EGL_HEIGHT specify the *logical* width and
> height of the buffer in pixels. For pixel formats with sub-sampled Chroma
> values, it should be trivial for the EGL implementation to calculate the
> width/height of the Chroma sample buffers using the logical width & height
> and by inspecting the pixel format passed as the EGL_LINUX_DRM_FOURCC_EXT
> attribute. I.e. If the pixel format says it's YUV420, the Chroma buffer's
> width = EGL_WIDTH/2 & height =EGL_HEIGHT/2.
>
> 11. How are Bayer formats handled?
>
> ANSWER: As of Linux 2.6.34, drm_fourcc.h does not include any Bayer formats.
> However, future kernel versions may add such formats in which case they
> would be handled in the same way as any other format.
>
> 12. Should the spec support buffers which have samples in a "narrow range"?
>
> Content sampled from older analogue sources typically don't use the full
> (0-256) range of the data type storing the sample and instead use a narrow
> (16-235) range to allow some headroom & toeroom in the signals to avoid
> clipping signals which overshoot slightly during processing. This is
> sometimes known as signals using "studio swing".
>
> ANSWER: Add a new attribute to define if the samples use a narrow 16-235
> range or the full 0-256 range.
>
> 13. Specifying the color space and range seems cumbersome, why not just
> allow the application to specify the full YUV->RGB color conversion matrix?
>
> ANSWER: Some hardware may not be able to use an arbitrary conversion matrix
> and needs to select an appropriate pre-defined matrix based on the color
> space and the sample range.
>
> 14. How do you handle EGL implementations which have restrictions on pitch
> and/or offset?
>
> ANSWER: Buffers being imported using dma_buf pretty much have to be
> allocated by a kernel-space driver. As such, it is expected that a system
> integrator would make sure all devices which allocate buffers suitable for
> exporting make sure they use a pitch supported by all possible importers.
> However, it is still possible eglCreateImageKHR can fail due to an
> unsupported pitch. Added a new error to the list indicating this.
>
> 15. Should this specification also describe how to export an existing
> EGLImage as a dma_buf file descriptor?
>
> ANSWER: No. Importing and exporting buffers are two separate operations and
> importing an existing dma_buf fd into an EGLImage is useful functionality in
> itself. Agree that exporting an EGLImage as a dma_buf fd is useful, E.g. it
> could be used by an OpenMAX IL implementation's OMX_UseEGLImage function to
> give access to the buffer backing an EGLImage to video hardware. However,
> exporting can be split into a separate extension specification.
>
>
> Revision History
>
> #4 (Tom Cooksey, October 04, 2012)
> - Fixed issue numbering!
> - Added issues 8 - 15.
> - Promoted proposal for Issue 3 to be the answer.
> - Added an additional attribute to allow an application to specify the color
> space as a hint which should address issue 4.
> - Added an additional attribute to allow an application to specify the chroma
> siting as a hint which should address issue 5.
> - Added an additional attribute to allow an application to specify the sample
> range as a hint which should address the new issue 12.
> - Added language to end of error section clarifying who owns the fd passed
> to eglCreateImageKHR if an error is generated.
>
> #3 (Tom Cooksey, August 16, 2012)
> - Changed name from EGL_EXT_image_external and re-written language to
> explicitly state this for use with Linux & dma_buf.
> - Added a list of issues, including some still open ones.
>
> #2 (Jesse Barker, May 30, 2012)
> - Revision to split eglCreateImageKHR functionality from export
> Functionality.
> - Update definition of EGLNativeBufferType to be a struct containing a list
> of handles to support multi-buffer/multi-planar formats.
>
> #1 (Jesse Barker, March 20, 2012)
> - Initial draft.
>
>
>
>
> _______________________________________________
> mesa-dev mailing list
> mesa-dev at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/mesa-dev
More information about the dri-devel
mailing list