[Pixman] [ARM] stack corruption in pixman 0.30.2

[Patrik, Kluba]

Hi!

I am playing with WebKitDFB on an ARMv6 platform, and have faced a
strange segmentation fault with PC/LR being 0x8/0x6, obviously meaning
control was transferred to NULL pointer somewhere. With a bit of luck
and gdb magic, I could track it down to
pixman_composite_src_8_8_asm_armv6(). Using breakpoints it was
confirmed to cause the problem.
That function comes from pixman-arm-simd*.*, the SOC being used has an
ARM1176JZF-S core, so all the required instructions are supported.

Based on the disassembly, pixman_composite_src_8_8_asm_armv6() has very
little to do with the stack. In the prologue it pushes several
registers, and in the epilogue it pops them. Besides that 2 registers are
pushed on the stack in the middle, and popped later. Jumps are made all
over the place, so based on conditions it could happen that this push and
pop can get out of sync. If the logic is good, then this will not happen,
just a theoretical possibility.

Before the epilogue (final pop), there's an unconditional SP+=8, which is
skipped from some places above by jumping to the instruction following it
(final pop). This is presumably to counter the 2-register-push before without
doing the 2-register-pop, in case.

But for me, that SP+=8 happens even when it should not, ruining the
stack frame. The mentioned 2-register-push/2-register-pop instructions are
not even executed.

I've tried to decipher the heavy macro usage in C and assembly, but failed.

It's possible that the root of the problem lies somewhere above, and
bad parameters reach pixman_composite_src_8_8_asm_armv6(), leading to this
situation, but I don't know how this could be tracked down.

Could somebody give me a hand, please?

A disassembly of the function can be found at:
http://pastebin.ca/2454215

A captured instruction flow can be found at:
http://pastebin.ca/2454212

A more detailed instruction flow with all register values for each
instruction (different execution but same flow) can be found at:
(gzipped txt)
http://ttb.li/fvthv

Thanks,
Patrik

PS: CC me, as I am not on the list

-- 
Patrik KLUBA
Software Developer at

DENSION Audio Systems Ltd.
H-1116 Budapest, Sztregova u. 1
Phone: +36 1 463 0470
Fax:   +36 1 463 0479
Web:   www.dension.com