[PATCH] drm/fbdev: Clamp fbdev surface size if too large

[Thomas Zimmermann]

Clamp the fbdev surface size of the available maximum height to avoid
failing to init console emulation. An example error is shown below.

  bad framebuffer height 2304, should be >= 768 && <= 768
  [drm] Initialized simpledrm 1.0.0 20200625 for simple-framebuffer.0 on minor 0
  simple-framebuffer simple-framebuffer.0: [drm] *ERROR* fbdev: Failed to setup generic emulation (ret=-22)

This is especially a problem with drivers that have very small screen
sizes and cannot over-allocate at all.

Signed-off-by: Thomas Zimmermann <tzimmermann at suse.de>
Fixes: 11e8f5fd223b ("drm: Add simpledrm driver")
Reported-by: Amanoel Dawod <kernel at amanoeldawod.com>
Reported-by: Zoltán Kővágó <dirty.ice.hu at gmail.com>
Reported-by: Michael Stapelberg <michael+lkml at stapelberg.ch>
Cc: Daniel Vetter <daniel.vetter at ffwll.ch>
Cc: Maxime Ripard <maxime at cerno.tech>
Cc: dri-devel at lists.freedesktop.org
Cc: <stable at vger.kernel.org> # v5.14+
---
 drivers/gpu/drm/drm_fb_helper.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/gpu/drm/drm_fb_helper.c b/drivers/gpu/drm/drm_fb_helper.c
index 6860223f0068..364f11900b37 100644
--- a/drivers/gpu/drm/drm_fb_helper.c
+++ b/drivers/gpu/drm/drm_fb_helper.c
@@ -1508,6 +1508,7 @@ static int drm_fb_helper_single_fb_probe(struct drm_fb_helper *fb_helper,
 {
 	struct drm_client_dev *client = &fb_helper->client;
 	struct drm_device *dev = fb_helper->dev;
+	struct drm_mode_config *config = &dev->mode_config;
 	int ret = 0;
 	int crtc_count = 0;
 	struct drm_connector_list_iter conn_iter;
@@ -1665,6 +1666,11 @@ static int drm_fb_helper_single_fb_probe(struct drm_fb_helper *fb_helper,
 	/* Handle our overallocation */
 	sizes.surface_height *= drm_fbdev_overalloc;
 	sizes.surface_height /= 100;
+	if (sizes.surface_height > config->max_height) {
+		drm_warn(dev, "Fbdev over-allocation too large; clamping height to %d\n",
+			 config->max_height);
+		sizes.surface_height = config->max_height;
+	}

 	/* push down into drivers */
 	ret = (*fb_helper->funcs->fb_probe)(fb_helper, &sizes);
--
2.33.0