Issues in current VESA driver.Color distortion when switch VT.

[Liu, Wolke]

Hi all,

Here is the bug I found in current vesa driver, could you have a look and give some advice? 3 patches are list in this mail, if ok, who can help to check in them to git depot?

1. Logic error
	As mentioned in another mail, this bug will cause color distortion in 8bpp when switch VT.
	Here is the fix: 

diff --git a/src/vesa.c b/src/vesa.c
index e4e6547..57ca3a7 100644
--- a/src/vesa.c
+++ b/src/vesa.c
@@ -1169,7 +1169,7 @@ VESASetMode(ScrnInfoPtr pScrn, DisplayModePtr pMode)
        VBESetLogicalScanline(pVesa->pVbe, pScrn->displayWidth);
 
     if (pScrn->bitsPerPixel == 8 && pVesa->vbeInfo->Capabilities[0] & 0x01 &&
-        !(data->data->MemoryModel & 0x6 || data->data->MemoryModel & 0x7)) 
+        !(data->data->MemoryModel == 0x6 || data->data->MemoryModel == 0x7)) 
        VBESetGetDACPaletteFormat(pVesa->pVbe, 8);
 
     pScrn->vtSema = TRUE;

2. Access VGA registers when in a non VGA compatible mode
　　This is conflict to VBE spec. I got color corruption when switch VT. In fact, it may have other side effect. Please see
　　http://cvsweb.xfree86.org/cvsweb/xc/programs/Xserver/hw/xfree86/drivers/vesa/vesa.c.diff?r1=text&tr1=1.47&r2=text&tr2=1.46&diff_format=l
      
      A.  Handle screen blanking in the vesa driver for non-VGA compatible modes  (David Dawes).

diff -u -r1.46 -r1.47
--- xc/programs/Xserver/hw/xfree86/drivers/vesa/vesa.c	2005/02/11 19:45:57	1.46
+++ xc/programs/Xserver/hw/xfree86/drivers/vesa/vesa.c	2005/02/26 03:18:38	1.47

@@ -1518,6 +1524,18 @@
     ScrnInfoPtr pScrn = xf86Screens[pScreen->myNum];
     VESAPtr pVesa = VESAGetRec(pScrn);
     Bool on = xf86IsUnblank(mode);
+    VbeModeInfoBlock *vMode;
+
+    if (pScrn->currentMode) {
+	vMode = ((VbeModeInfoData*)pScrn->currentMode->Private)->data;
+	/*
+	 * Don't try to use VGA blanking for modes that are not
+	 * VGA-compatible.
+	 */
+	if (vMode->ModeAttributes & (1 << 5)) {
+	    return FALSE;
+	}
+    }
 
     if (on)
 	SetTimeSinceLastInputEvent();

Though my cards works well without above patch, I think this patch is needed.
　
　　B. the same root cause will affect palette too in VESALoadPalette(), it will cause color distortion now.
diff --git a/src/vesa.c b/src/vesa.c
index e4e6547..dbb70bc 100644
--- a/src/vesa.c
+++ b/src/vesa.c
@@ -1335,7 +1335,10 @@ VESALoadPalette(ScrnInfoPtr pScrn, int numColors, int *indices,
 {
     VESAPtr pVesa = VESAGetRec(pScrn);
     int i, idx;
+   VbeModeInfoBlock *data = ((VbeModeInfoData*)(pScrn->currentMode->Private))->data;
 
+    if (data->ModeAttributes & 0x20)
+        return;
 #if 0
 
     /* This code works, but is very slow for programs that use it intensively */
　　
　　This fix is workable. Only the mode is VGA compatible, we can reload the palette via RAMDAC register.
　　If is not a VGA compatible mode, in fact, we should use the commented git code. But this method will cause color distortion in some card. The reason is the system palette table is different to different hardware palette.  I have compared NV17 and RV620 card, they are different. There are codes in the git driver folders that contain the converting algorithm for different chips. So it beyond the vesa driver's capability that to convert system palette to correct H/W palette format. I just jump over it and use the default H/W palette. Most cards will use the default h/w palette after set mode. It works on my cards, but I am not sure it will affect others.

The long history of VESA driver beyond my knowledge, please correct me. Thanks for you attention.
　　
Regards,
Wolke