cmap problems with aty128fb.c

Paul Mackerras (paulus@samba.org)
Wed, 8 May 2002 18:40:24 +1000 (EST)


I have been having difficulties with aty128fb.c in 2.5.14. The
version in Linus' tree crashes on boot for me with a null pointer
dereference in memcpy, called from fb_copy_cmap from gen_set_cmap,
from fbcon_show_logo via the fbops->fb_set_cmap pointer.

At the lowest level the null pointer dereference is caused by a
signed/unsigned bug in fb_copy_cmap. We are ending up with size > 0
even when to->start and to->len were both zero. The patch below fixes
that.

At the next level up, the problem seems to be that info->cmap is never
getting initialized. How and where is it supposed to be initialized?

Paul.

diff -urN linux-2.5/drivers/video/fbcmap.c pmac-2.5/drivers/video/fbcmap.c
--- linux-2.5/drivers/video/fbcmap.c Mon Apr 29 16:25:24 2002
+++ pmac-2.5/drivers/video/fbcmap.c Wed May 8 16:29:04 2002
@@ -150,9 +150,9 @@
else
tooff = from->start-to->start;
size = to->len-tooff;
- if (size > from->len-fromoff)
+ if (size > (int)(from->len-fromoff))
size = from->len-fromoff;
- if (size < 0)
+ if (size <= 0)
return;
size *= sizeof(u16);

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/