[Swfdec] VP6 issues in Swfdec

Wed Sep 5 04:27:31 PDT 2007

Hi,

I'm one of the Swfdec[1] Flash player developers. As you may be aware,
Flash uses VP6 as a possible video decoder.
Someone recently grabbed all of the videos at http://pown.alluc.org
and threw them at ffmpeg. Surprisingly, lots of them killed the vp6
decoder and in turn my browser.

I'll attached a series of patches for issues that I could fix myself,
and point out issues I could not fix myself.
If you want to reproduce the issues, you'll have to get swfdec git,
wget the files I'll point to and try to play them with player/swfplay
from the swfdec sources.

-- SEGV
http://pown.alluc.org/179.swf
patch: check_coeff_offset.diff
SInce this was the first file where I hit the issue, I'll just list
this one. FFmpeg reads an offset and doesn't validate it, this patch
does that.

-- sanity checks
patch: sanity.diff
While trying to find the issue, I added sanity checks to the read
functions used by vp56, as there doesn't seem to be any protection
against overreads.

-- small memleak
patch: small_memleak.diff
This is a small memleak fix I noticed when looking at the code.

-- memleak
http://pown.alluc.org/210.swf
http://pown.alluc.org/991.swf
There's a huge memleak reported by valgrind (this is from a run of 210.swf):
==31127== 224,784,144 bytes in 177 blocks are still reachable in loss
record 5,098 of 5,098
==31127==    at 0x4021990: memalign (vg_replace_malloc.c:332)
==31127==    by 0x510ECE6: av_malloc (mem.c:61)
==31127==    by 0x4D71B01: avcodec_default_get_buffer (utils.c:308)
==31127==    by 0x4F4F789: vp56_decode_frame (vp56.c:509)
==31127==    by 0x4D70B7D: avcodec_decode_video (utils.c:937)
==31127==    by 0x406712D: swfdec_video_decoder_ffmpeg_decode
(swfdec_codec_ffmpeg.c:235)
==31127==    by 0x406A2A8: swfdec_video_decoder_decode
(swfdec_codec_video.c:102)
==31127==    by 0x40A32F0: swfdec_video_input_iterate (swfdec_video.c:82)
==31127==    by 0x40A40F8: swfdec_video_movie_iterate_end
(swfdec_video_movie.c:92)
==31127==    by 0x4089287: swfdec_player_iterate (swfdec_player.c:1114)
==31127==    by 0x4089533: swfdec_player_do_advance (swfdec_player.c:1156)
==31127==    by 0x407A53A: swfdec_marshal_VOID__ULONG_UINT
(swfdec_marshal.c:246)

-- "alternative entropy decoding not supported"
http://pown.alluc.org/179.swf
http://pown.alluc.org/185.swf
http://pown.alluc.org/207.swf
http://pown.alluc.org/243.swf
http://pown.alluc.org/376.swf
http://pown.alluc.org/453.swf
http://pown.alluc.org/498.swf
http://pown.alluc.org/652.swf
http://pown.alluc.org/653.swf
http://pown.alluc.org/658.swf
http://pown.alluc.org/661.swf
http://pown.alluc.org/673.swf
http://pown.alluc.org/756.swf
http://pown.alluc.org/845.swf
http://pown.alluc.org/850.swf
http://pown.alluc.org/860.swf
http://pown.alluc.org/962.swf
http://pown.alluc.org/966.swf
http://pown.alluc.org/990.swf
http://pown.alluc.org/1040.swf
http://pown.alluc.org/1063.swf
http://pown.alluc.org/1064.swf
http://pown.alluc.org/1106.swf
http://pown.alluc.org/1126.swf
The VP6 decoder often claims the above error message and then produces
weird images. So feel free to use the files listed here if you want to
implement it. I should also notice that these files do a lot of other
weird stuff (like using invalid image sizes for the decoded image). I
guess that's just a side effect?

-- SEGV in mmx code
http://pown.alluc.org/280.swf
http://pown.alluc.org/416.swf
http://pown.alluc.org/497.swf
http://pown.alluc.org/830.swf
There's a SEGV here. I have no clue where it comes from. I'll leave
that for you to figure out:
#0  0xb6f69cfa in put_pixels8_mmx (block=0xb6a827a8 "��������ds\t",
pixels=0xb611d6a8 '�' <repeats 200 times>..., line_size=1464, h=8)
    at i386/dsputil_mmx.c:416
#1  0xb71431d6 in vp56_decode_frame (avctx=0x80d6c30, data=0x80d24a0,
data_size=0xbfa4b008,
    buf=0x8114980
"�\212����r9qy\234#]�\003��Q�]\214\216m��'\033g�\037��\200�\034�`wm<j�\221\\\f\025?�6�\220\032\026Lp\232j\034I�;��U�\203�\230yPA\226*Ȱ^\001�\n\002�\221��\"�5H7Q����\236�<\026�\212M{\236��\231sQ\201+{\214\226�,��-�)\001�\017�\031��B�\a*|9�~\b\202g>z�ow,��\032^
�x<��\237?�\006bn�Q�\0044S8\033�#\214T-���=\202�zͥ�\231\n\201\234�;\\�OF�\006ж�\v�"...,
buf_size=2840) at vp56.c:432
#2  0xb6f63b7e in avcodec_decode_video (avctx=0x80d6c30,
picture=0x80d24a0, got_picture_ptr=0xbfa4b008,
    buf=0x8114980
"�\212����r9qy\234#]�\003��Q�]\214\216m��'\033g�\037��\200�\034�`wm<j�\221\\\f\025?�6�\220\032\026Lp\232j\034I�;��U�\203�\230yPA\226*Ȱ^\001�\n\002�\221��\"�5H7Q����\236�<\026�\212M{\236��\231sQ\201+{\214\226�,��-�)\001�\017�\031��B�\a*|9�~\b\202g>z�ow,��\032^
�x<��\237?�\006bn�Q�\0044S8\033�#\214T-���=\202�zͥ�\231\n\201\234�;\\�OF�\006ж�\v�"...,
buf_size=2840) at utils.c:937

Cheers,
Benjamin

[1] http://swfdec.freedesktop.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: check_coeff_offset.diff
Type: text/x-patch
Size: 551 bytes
Desc: not available
Url : http://lists.freedesktop.org/archives/swfdec/attachments/20070905/a5055612/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: small_memleak.diff
Type: text/x-patch
Size: 440 bytes
Desc: not available
Url : http://lists.freedesktop.org/archives/swfdec/attachments/20070905/a5055612/attachment-0001.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sanity.diff
Type: text/x-patch
Size: 2404 bytes
Desc: not available
Url : http://lists.freedesktop.org/archives/swfdec/attachments/20070905/a5055612/attachment-0002.bin 