VT switch broken with docking station DP

[Ville Syrjälä]

On Thu, Jan 05, 2017 at 04:37:27PM +0100, Takashi Iwai wrote:
> Hi,
> 
> recently I noticed that VT console doesn't work any longer when I dock
> a Dell E7270 laptop with a DP monitor.  The bug detail is like this:
> 
> At first, I boot the laptop without dock.  I can switch between X and
> VT via ctrl-alt-F1, so far.  Then I dock it to a docking station
> connected with a DP monitor.  Now, when I switch to VT, it behaves as
> if frozen, the X graphics screen remains.  But actually it's only
> graphics and the keyboard input is processed in VT.  I can go back to
> X via alt-F7 again.  The situation remains until I undock and I kill X
> once.
> 
> After looking more deeply at drm debug log, I found out that it's
> caused by the drm atomic check.  Essentially, it's because eDP has the
> lower resolution (1366x768) than DP (1920x1080).  Since booting with
> eDP, the frame buffer size is 1366x768.  Then it hits the following
> check in drm_atomic_plane_check():
> 
> 	fb_width = state->fb->width << 16;
> 	fb_height = state->fb->height << 16;
> 
> 	/* Make sure source coordinates are inside the fb. */
> 	if (state->src_w > fb_width ||
> 	    state->src_x > fb_width - state->src_w ||
> 	    state->src_h > fb_height ||
> 	    state->src_y > fb_height - state->src_h) {
> 		DRM_DEBUG_ATOMIC("Invalid source coordinates "
> 				 "%u.%06ux%u.%06u+%u.%06u+%u.%06u\n",
> 				 state->src_w >> 16, ((state->src_w & 0xffff) * 15625) >> 10,
> 				 state->src_h >> 16, ((state->src_h & 0xffff) * 15625) >> 10,
> 				 state->src_x >> 16, ((state->src_x & 0xffff) * 15625) >> 10,
> 				 state->src_y >> 16, ((state->src_y & 0xffff) * 15625) >> 10);
> 		return -ENOSPC;
> 	}
> 
> Actually after commenting out "return -ENOSPC", VT switch works fine.
> 
> But the code above made me wonder what's the requirement here.  IIRC,
> the VT always worked on a display with a higher resolution even if the
> frame buffer is smaller.  Only a part of display was used, but it was
> OK, far better than the frozen graphics :)
> 
> Can we simply drop this check, or may we add a flag to skip it for VT
> switching?  Or any better idea?

Find out why it didn't allocate a big enough framebuffer to begin with,
or alternatively why it tried to specify source coordinates exceeding
the fb dimensions.

There is clearly a bug somewhere, just not here.

-- 
Ville Syrjälä
Intel OTC