[PATCH] render: Don't generate invalid pixman format when using a 24bpp framebuffer with a 32bit depth visual.

[Takashi Iwai]

At Fri, 6 Jun 2014 16:49:58 +1200,
Robert Ancell wrote:
> 
> Some background on this patch:
> 
> I've been trying to investigate why gnome-terminal doesn't render
> correctly when running Ubuntu 14.04 LTS inside qemu (on Ubuntu 14.10).
> The window is compressed in the X axis and has transparency (seems to
> be a buffer somewhere that is being written in RGB and read as RGBA).
> 
> The issue seems to be GTK+ applications using the RGBA visual when
> qemu/X/fb is providing a depth 24 root window. Playing around with
> test cases showed the assertion mentioned in the patch. I'm not sure
> if fb should be allowed to have a depth greater than the bpp and if
> the bug is actually there but this patch allows correct rendering in
> my test case now. I guess the Picture is being converted from RGBA to
> RGB when it hits the root window?
> 
> Note this patch doesn't fix the gnome-terminal rendering issue, so
> I'll continue down the rabbit hole. Any suggestions as to the causes
> of this are greatly appreciated.
> 
> Attached is a program that doesn't render on the current master and
> does with the applied patch.

Tested-by: Takashi Iwai <tiwai at suse.de>

Please someone review and take this, too.


thanks,

Takashi


> 
> --Robert
> 
> On Fri, Jun 6, 2014 at 4:36 PM, Robert Ancell
> <robert.ancell at canonical.com> wrote:
> > When using the fb backend at 24bpp it allows a visual with 32 bit depth.
> > This would cause RENDER to try and create an invalid pixman buffer and hit the
> > following assertion when trying to render to it:
> >
> > *** BUG ***
> > In create_bits_image_internal: The expression PIXMAN_FORMAT_BPP (format) >= PIXMAN_FORMAT_DEPTH (format) was false
> > Set a breakpoint on '_pixman_log_error' to debug
> >
> > Fix is to ensure that the bpp is always at least as big as the drawable depth.
> > ---
> >  render/picture.c | 6 +++++-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
> >
> > diff --git a/render/picture.c b/render/picture.c
> > index 7da9310..f411521 100644
> > --- a/render/picture.c
> > +++ b/render/picture.c
> > @@ -762,6 +762,7 @@ CreatePicture(Picture pid,
> >  {
> >      PicturePtr pPicture;
> >      PictureScreenPtr ps = GetPictureScreen(pDrawable->pScreen);
> > +    int bpp;
> >
> >      pPicture = dixAllocateScreenObjectWithPrivates(pDrawable->pScreen,
> >                                                     PictureRec, PRIVATE_PICTURE);
> > @@ -773,7 +774,10 @@ CreatePicture(Picture pid,
> >      pPicture->id = pid;
> >      pPicture->pDrawable = pDrawable;
> >      pPicture->pFormat = pFormat;
> > -    pPicture->format = pFormat->format | (pDrawable->bitsPerPixel << 24);
> > +    bpp = pDrawable->bitsPerPixel;
> > +    if (bpp < pFormat->depth)
> > +        bpp = BitsPerPixel (pFormat->depth);
> > +    pPicture->format = pFormat->format | (bpp << 24);
> >
> >      /* security creation/labeling check */
> >      *error = XaceHook(XACE_RESOURCE_ACCESS, client, pid, PictureType, pPicture,
> > --
> > 2.0.0
> >