[PATCH 1/2] ephyr: hostx_screen_init(): Fix bits_per_pixel and bytes_per_line

[Michele Baldessari]

On Mon, Oct 21, 2013 at 05:18:45PM -0400, Søren Sandmann wrote:
> When the depth of the Xephyr server matches that of the host X server,
> Xephyr simply uses the buffer associated with the XImage as its
> framebuffer. In this case, it is correct to get the bits_per_pixel and
> bytes_per_line values returned from hostx_screen_init() from the XImage.
> 
> However, when the depth doesn't match the host, Xephyr uses a private
> framebuffer that is periodically copied to the XImage. In this case,
> the returned values of bits_per_pixel and bytes_per_line should be
> those of the private framebuffer, not those of the XImage.
> ---
>  hw/kdrive/ephyr/hostx.c |    9 ++++++---
>  1 files changed, 6 insertions(+), 3 deletions(-)
> 
> diff --git a/hw/kdrive/ephyr/hostx.c b/hw/kdrive/ephyr/hostx.c
> index d985571..86f3679 100644
> --- a/hw/kdrive/ephyr/hostx.c
> +++ b/hw/kdrive/ephyr/hostx.c
> @@ -693,9 +693,6 @@ hostx_screen_init(KdScreenInfo *screen,
>              malloc(scrpriv->ximg->stride * buffer_height);
>      }
>  
> -    *bytes_per_line = scrpriv->ximg->stride;
> -    *bits_per_pixel = scrpriv->ximg->bpp;
> -
>      if (scrpriv->win_pre_existing == None && !EphyrWantResize) {
>          /* Ask the WM to keep our size static */
>          xcb_size_hints_t size_hints = {0};
> @@ -715,10 +712,16 @@ hostx_screen_init(KdScreenInfo *screen,
>      scrpriv->win_height = height;
>  
>      if (host_depth_matches_server(scrpriv)) {
> +        *bytes_per_line = scrpriv->ximg->stride;
> +        *bits_per_pixel = scrpriv->ximg->bpp;
> +
>          EPHYR_DBG("Host matches server");
>          return scrpriv->ximg->data;
>      }
>      else {
> +        *bytes_per_line = width * (scrpriv->server_depth >> 3);
> +        *bits_per_pixel = scrpriv->server_depth;
> +
>          EPHYR_DBG("server bpp %i", scrpriv->server_depth >> 3);
>          scrpriv->fb_data =
>              malloc(width * buffer_height * (scrpriv->server_depth >> 3));

Thanks a lot Søren, 

this fixes it for me for both the normal case and the resizeable case.
I noticed that if I launch DISPLAY=:1 xclock on Xephyr -screen 320x200x8 :1 and then
kill xclock, I get the following backtrace:
ephyrScreenInitialize (screen=0x823d50) at ephyr.c:124
124                 scrpriv->server_depth = screen->fb.depth;

#0  ephyrScreenInitialize (screen=0x823d50) at ephyr.c:124
#1  0x000000000048c1b7 in KdInitScreen (pScreenInfo=pScreenInfo at entry=0x81ec80 <screenInfo>, screen=screen at entry=0x823d50, 
    argc=argc at entry=4, argv=argv at entry=0x7fffffffe0d8) at kdrive.c:946
#2  0x000000000048c281 in KdInitOutput (pScreenInfo=pScreenInfo at entry=0x81ec80 <screenInfo>, argc=argc at entry=4, 
    argv=argv at entry=0x7fffffffe0d8) at kdrive.c:1086
#3  0x0000000000422726 in InitOutput (pScreenInfo=pScreenInfo at entry=0x81ec80 <screenInfo>, argc=argc at entry=4, 
    argv=argv at entry=0x7fffffffe0d8) at ephyrinit.c:80
#4  0x000000000043e04b in dix_main (argc=4, argv=0x7fffffffe0d8, envp=<optimized out>) at main.c:200
#5  0x0000003714221b75 in __libc_start_main (main=0x41fad0 <main>, argc=4, ubp_av=0x7fffffffe0d8, init=<optimized out>, 
    fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe0c8) at libc-start.c:274
#6  0x000000000041fb01 in _start ()

The issue is that after killing the X client we have:
(gdb) p screen->driver
$1 = (void *) 0x0

EphyrScrPriv *scrpriv = screen->driver;

hth,
Michele
-- 
Michele Baldessari            <michele at acksyn.org>
C2A5 9DA3 9961 4FFB E01B  D0BC DDD4 DCCB 7515 5C6D