[RFC][PATCH 1/2] drm_hwcomposer: Error out on YUV layer as it would fail for single planes

[Robert Foss]

Hey John,

On 03/07/2018 12:19 AM, John Stultz wrote:
> As suggested by Alexandru-Cosmin Gheorghe:
> 
> ConvertHALFormatToDrm logic would work only for 1 plane formats,
> and probably gets rejected by drmModeAddFb2, but to save
> debugging time  maybe it worth removing DRM_FORMAT_YVU420 from
> ConvertHALFormatToDrm and checking it's return code.
> 
> So this patch tries to do this.
> 
> Cc: Marissa Wall <marissaw at google.com>
> Cc: Sean Paul <seanpaul at google.com>
> Cc: Dmitry Shmidt <dimitrysh at google.com>
> Cc: Robert Foss <robert.foss at collabora.com>
> Cc: Matt Szczesiak <matt.szczesiak at arm.com>
> Cc: Liviu Dudau <Liviu.Dudau at arm.com>
> Cc: David Hanna <david.hanna11 at gmail.com>
> Cc: Rob Herring <rob.herring at linaro.org>
> Cc: Alexandru-Cosmin Gheorghe <Alexandru-Cosmin.Gheorghe at arm.com>
> Signed-off-by: John Stultz <john.stultz at linaro.org>
> ---
>   platformdrmgeneric.cpp | 15 +++++++++++----
>   1 file changed, 11 insertions(+), 4 deletions(-)
> 
> diff --git a/platformdrmgeneric.cpp b/platformdrmgeneric.cpp
> index 741d42b..33f1ea0 100644
> --- a/platformdrmgeneric.cpp
> +++ b/platformdrmgeneric.cpp
> @@ -76,8 +76,6 @@ uint32_t DrmGenericImporter::ConvertHalFormatToDrm(uint32_t hal_format) {
>         return DRM_FORMAT_ABGR8888;
>       case HAL_PIXEL_FORMAT_RGB_565:
>         return DRM_FORMAT_BGR565;
> -    case HAL_PIXEL_FORMAT_YV12:
> -      return DRM_FORMAT_YVU420;

I'm not sure I understand the rationale for removing YVU420.

>       default:
>         ALOGE("Cannot convert hal format to drm format %u", hal_format);
>         return -EINVAL;
> @@ -88,10 +86,15 @@ EGLImageKHR DrmGenericImporter::ImportImage(EGLDisplay egl_display, buffer_handl
>     gralloc_drm_handle_t *gr_handle = gralloc_drm_handle(handle);
>     if (!gr_handle)
>       return NULL;
> +
> +  EGLint fmt = ConvertHalFormatToDrm(gr_handle->format);
> +  if (fmt < 0)
> +	return NULL;
> +
>     EGLint attr[] = {
>       EGL_WIDTH, gr_handle->width,
>       EGL_HEIGHT, gr_handle->height,
> -    EGL_LINUX_DRM_FOURCC_EXT, (EGLint)ConvertHalFormatToDrm(gr_handle->format),
> +    EGL_LINUX_DRM_FOURCC_EXT, fmt,
>       EGL_DMA_BUF_PLANE0_FD_EXT, gr_handle->prime_fd,
>       EGL_DMA_BUF_PLANE0_OFFSET_EXT, 0,
>       EGL_DMA_BUF_PLANE0_PITCH_EXT, gr_handle->stride,
> @@ -112,10 +115,14 @@ int DrmGenericImporter::ImportBuffer(buffer_handle_t handle, hwc_drm_bo_t *bo) {
>       return ret;
>     }
>   
> +  uint32_t  fmt = ConvertHalFormatToDrm(gr_handle->format);
> +  if (fmt < 0)
> +        return fmt;
> +
>     memset(bo, 0, sizeof(hwc_drm_bo_t));
>     bo->width = gr_handle->width;
>     bo->height = gr_handle->height;
> -  bo->format = ConvertHalFormatToDrm(gr_handle->format);
> +  bo->format = fmt;
>     bo->usage = gr_handle->usage;
>     bo->pitches[0] = gr_handle->stride;
>     bo->gem_handles[0] = gem_handle;
>